Loading ...
Sorry, an error occurred while loading the content.

Re: Python IDE-ish features (was: Vim 7.0 sandbox changes break a lot of my configuration)

Expand Messages
  • Mikolaj Machowski
    ... That could be partially solved by omnicompletion providing some kind of structure. I was thinking about JavaScript but there you have very little to
    Message 1 of 21 , Jan 6, 2006
    • 0 Attachment
      Dnia czwartek, 5 stycznia 2006 22:41, G. Sumner Hayes napisał:
      > and I try to complete, I have no idea what the types
      > of x and y are. Indeed, one place in the code might
      > call cmp with x and y as DollarValues and another
      > might call it with TestScores (and this kind of
      > automatic polymorphism is a strong suit of languages
      > like Python, Ruby, Smalltalk, etc).
      >
      > Or I could have:
      > aClass = loadClass("happyObject")
      > a=aClass()
      > a.^X^O
      >
      > Same idea; loadClass could return any class at all.
      >
      > Now, for some things it _might_ be more obvious. But
      > a reasonable solution would mean parsing the entire
      > project to see what values are actually being passed
      > around and doing some amount of type propogation.
      > Probably means a ctags-style type inference engine,
      > and even then it's not perfect in cases like
      > input = sys.read()
      > a = exec(input)

      That could be partially solved by omnicompletion providing some kind of
      structure. I was thinking about JavaScript but there you have very
      little to differentiate between variable/object name, function name and
      DOM elements. In some cases you can guess but not ever be near 95% you
      mentioned.

      If omnicompletion could take also Dictionaries as argument and present
      them as structure...

      m.
    • G. Sumner Hayes
      Note: I think I may have found a memory leak in vim7, see below. ... note). [SNIP] ... After looking at it a bit more, it seems like the function I need is
      Message 2 of 21 , Jan 6, 2006
      • 0 Attachment
        Note: I think I may have found a memory leak in vim7,
        see below.

        Bram Moolenaar <Bram@...> wrote:
        > G. Sumner Hayes wrote:
        > > I've attached a proof-of-concept patch against
        > > that (not ready for general use, see problem
        note).
        [SNIP]
        > > PROBLEM: it doesn't free up the list returned; I'm
        > > very unclear on how the memory management in vim
        > > works.

        > I'm trying to keep most manipulation with script
        > variables local to eval.c.

        After looking at it a bit more, it seems like the
        function I need is exported already (clear_tv).

        2 questions:
        1. If I do clear_tv(my_tv), that should empty out any
        lists, strings, dicts, etc within the tv, right?
        2. assuming that my_tv was originally dynamically
        allocated via "my_tv = (typval_t *)
        alloc(sizeof(typval_T));" then I still need to call
        vim_free(my_tv) after the clear_tv call, right?

        If I'm right on both of those, then the last if block
        in VimFree from my patch should be changed by
        eliminating the "if(our_tv->v_type == VAL_STRING)"
        block and replacing it with "clear_tv(our_tv);"

        New diff attached. I think this is ready for other
        people to test (or to apply), as far as I can see the
        memory management is okay now. If someone else out
        there knows the if_python code, reading my diff
        wouldn't hurt.

        Bram, feel free to use/distribute under the standard
        vim license.

        POSSIBLE MEMORY LEAK: quickfix.c at line 2998 clears a
        tv that it got from eval_expr. Since eval_expr
        allocates the tv dynamically, I believe a line should
        be added after that "vim_free(tv);". Is that right?



        __________________________________________
        Yahoo! DSL – Something to write home about.
        Just $16.99/mo. or less.
        dsl.yahoo.com
      • G. Sumner Hayes
        ... Gah! Here it is. It uses clear_tv/vim_free, that could easily be changed to free_tv when you make that global. __________________________________________
        Message 3 of 21 , Jan 7, 2006
        • 0 Attachment
          Bram Moolenaar <Bram@...> wrote:
          > > New diff attached. I think this is ready for
          > > other people to test (or to apply), as far as I
          > > can see the memory management is okay now.
          > > If someone else out there knows the if_python
          > > code, reading my diff wouldn't hurt.
          >
          > You didn't attach a patch...

          Gah! Here it is. It uses clear_tv/vim_free, that
          could easily be changed to free_tv when you make that global.



          __________________________________________
          Yahoo! DSL – Something to write home about.
          Just $16.99/mo. or less.
          dsl.yahoo.com
        • Bram Moolenaar
          ... Very good. So it s completely functional now, right? I ll put it in the todo list to be included. But perhaps you can fix the formatting to match the Vim
          Message 4 of 21 , Jan 8, 2006
          • 0 Attachment
            G. Sumner Hayes wrote:

            > Bram Moolenaar <Bram@...> wrote:
            > > > New diff attached. I think this is ready for
            > > > other people to test (or to apply), as far as I
            > > > can see the memory management is okay now.
            > > > If someone else out there knows the if_python
            > > > code, reading my diff wouldn't hurt.
            > >
            > > You didn't attach a patch...
            >
            > Gah! Here it is. It uses clear_tv/vim_free, that
            > could easily be changed to free_tv when you make that global.

            Very good. So it's completely functional now, right?

            I'll put it in the todo list to be included. But perhaps you can fix
            the formatting to match the Vim source code. And I also need an update
            for the documentation.

            --
            'Well, here's something to occupy you and keep your mind off things.'
            'It won't work, I have an exceptionally large mind.'
            -- Douglas Adams, "The Hitchhiker's Guide to the Galaxy"

            /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
            /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
            \\\ download, build and distribute -- http://www.A-A-P.org ///
            \\\ help me help AIDS victims -- http://www.ICCF.nl ///
          • G. Sumner Hayes
            ... Yes, it looks functional to me. The taglist function returns a list of dictionaries keyed/valued by strings and so seems to test every case except
            Message 5 of 21 , Jan 9, 2006
            • 0 Attachment
              Bram Moolenaar <Bram@...> wrote:
              > Very good. So it's completely functional now,
              > right?

              Yes, it looks functional to me. The taglist function
              returns a list of dictionaries keyed/valued by strings
              and so seems to test every case except integers, which
              I've tested independently (and they do come back as
              strings, for backward compatibility).

              I've also done things like:
              let a = {1:'one', 2:['two', 'three']}
              py print vim.eval('a')[2] #and so forth
              to test nesting dicts/lists in the other direction.

              I haven't exactly tried throwing huge data structures
              at it but I see no reason it shouldn't work.

              > I'll put it in the todo list to be included. But
              > perhaps you can fix the formatting to match the Vim
              > source code. And I also need an update for the
              > documentation.

              I tried to fix formatting (mostly braces) to conform
              with the rest of the vim code. Attached. Most
              functions in that file have an /*ARGSUSED*/ comment
              (probably for lint or similar) that I don't know the
              exact meaning of and so did not duplicate.

              Also gave a shot at documentation updates. Attached.

              Open issues:
              1. What should we do if the eval doesn't return a
              supported type (I return None, presumably a vim
              exception will happen and I'm not sure that we want to
              raise a Python exception.)
              2. 4 lines are duplicated from eval.c, we might want
              to move at least the #defines into a public header and
              avoid duplication
              3. I use clear_tv/vim_free(tv) to free memory. You
              may want to switch that to free_tv when you publish
              that function.



              __________________________________________
              Yahoo! DSL – Something to write home about.
              Just $16.99/mo. or less.
              dsl.yahoo.com
            • Bram Moolenaar
              ... Well, you could try doing something complicated and see if Vim crashes. Recursive references (e.g., a dict containing itself) are always great fun. And
              Message 6 of 21 , Jan 9, 2006
              • 0 Attachment
                G. Sumner Hayes wrote:

                > Bram Moolenaar <Bram@...> wrote:
                > > Very good. So it's completely functional now,
                > > right?
                >
                > Yes, it looks functional to me. The taglist function
                > returns a list of dictionaries keyed/valued by strings
                > and so seems to test every case except integers, which
                > I've tested independently (and they do come back as
                > strings, for backward compatibility).
                >
                > I've also done things like:
                > let a = {1:'one', 2:['two', 'three']}
                > py print vim.eval('a')[2] #and so forth
                > to test nesting dicts/lists in the other direction.
                >
                > I haven't exactly tried throwing huge data structures
                > at it but I see no reason it shouldn't work.

                Well, you could try doing something complicated and see if Vim crashes.
                Recursive references (e.g., a dict containing itself) are always great
                fun.

                And when you have the ccmalloc library you could check for memory leaks
                (search the Makefile for -DEXITFREE).

                > > I'll put it in the todo list to be included. But
                > > perhaps you can fix the formatting to match the Vim
                > > source code. And I also need an update for the
                > > documentation.
                >
                > I tried to fix formatting (mostly braces) to conform
                > with the rest of the vim code. Attached. Most
                > functions in that file have an /*ARGSUSED*/ comment
                > (probably for lint or similar) that I don't know the
                > exact meaning of and so did not duplicate.
                >
                > Also gave a shot at documentation updates. Attached.

                Thanks for the update!

                > Open issues:
                > 1. What should we do if the eval doesn't return a
                > supported type (I return None, presumably a vim
                > exception will happen and I'm not sure that we want to
                > raise a Python exception.)

                Using None might be the best. Or transform it to a string with
                tv2string().

                > 2. 4 lines are duplicated from eval.c, we might want
                > to move at least the #defines into a public header and
                > avoid duplication

                This part is a bit problemous, knowledge about the Dictionary
                implemenation is being used. It's difficult to avoid though. Instead
                of the #define HI2DI you could use a function that is in eval.c.

                > 3. I use clear_tv/vim_free(tv) to free memory. You
                > may want to switch that to free_tv when you publish
                > that function.

                Yes, I made a note about that.

                - Bram

                --
                Nothing is fool-proof to a sufficiently talented fool.

                /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                \\\ download, build and distribute -- http://www.A-A-P.org ///
                \\\ help me help AIDS victims -- http://www.ICCF.nl ///
              • G. Sumner Hayes
                Thanks for looking at this Bram. ... Deep structures seem okay. ... These break. I m tempted to punt on properly handling recursive objects for now, and just
                Message 7 of 21 , Jan 9, 2006
                • 0 Attachment
                  Thanks for looking at this Bram.

                  Bram Moolenaar <Bram@...> wrote:
                  > G. Sumner Hayes wrote:
                  > Well, you could try doing something complicated and
                  > see if Vim crashes.

                  Deep structures seem okay.

                  > Recursive references (e.g., a dict containing
                  > itself) are always great fun.

                  These break. I'm tempted to punt on properly handling
                  recursive objects for now, and just put in a maximum
                  recursion depth (as Vim script does when you echo such
                  a structure); a proper fix requires keeping a table of
                  visited objects.

                  > And when you have the ccmalloc library you could
                  > check for memory leaks (search the Makefile for
                  > -DEXITFREE).

                  I don't have ccmalloc. I use dynamic memory in 2
                  ways:
                  1. The tv returned by eval_expr, which is always freed
                  unless it's NULL. Should be fine.
                  2. Python objects are allocated; these are all
                  reference counted and stored, so if the if_python
                  interface treated the returned objects properly before
                  then it should still be fine.

                  Obviously that's the theory, if someone out there has
                  ccmalloc then testing would be a good idea.

                  > > Open issues:
                  > > 1. What should we do if the eval doesn't return a
                  > > supported type (I return None, presumably a vim
                  > > exception will happen and I'm not sure that we
                  > > want to raise a Python exception.)
                  >
                  > Using None might be the best. Or transform it to a
                  > string with tv2string().

                  I don't like a string, since it makes it harder for
                  Python extensions to test whether the call failed. An
                  exception might be reasonable, but None is also
                  reasonable and already implemented so unless someone
                  has a good argument for the exception I'm inclined to
                  leave as-is.

                  > > 2. 4 lines are duplicated from eval.c, we might
                  > > want to move at least the #defines into a
                  > > public header and avoid duplication
                  >
                  > This part is a bit problemous, knowledge about the
                  > Dictionary implemenation is being used. It's
                  > difficult to avoid though. Instead of the #define
                  > HI2DI you could use a function that is in eval.c.

                  Good idea, I'll do that.

                  Thanks again for your time,

                  Sumner



                  __________________________________________
                  Yahoo! DSL – Something to write home about.
                  Just $16.99/mo. or less.
                  dsl.yahoo.com
                • G. Sumner Hayes
                  New patch attached. I m not re-sending the doc patch as it s unchanged. ... Done. Dealing properly with recursive objects is on my TODO but I want to look at
                  Message 8 of 21 , Jan 9, 2006
                  • 0 Attachment
                    New patch attached. I'm not re-sending the doc patch
                    as it's unchanged.

                    "G. Sumner Hayes" <sumnervim@...> wrote:
                    > Bram Moolenaar <Bram@...> wrote:
                    > > Recursive references (e.g., a dict containing
                    > > itself) are always great fun.
                    >
                    > These break. I'm tempted to punt on properly
                    > handling recursive objects for now, and just
                    > put in a maximum recursion depth (as Vim
                    > script does when you echo such a structure)

                    Done. Dealing properly with recursive objects is on
                    my TODO but I want to look at another issue first. So
                    for now I cap recursion.

                    > > HI2DI you could use a function that is in eval.c.
                    >
                    > Good idea, I'll do that.

                    Done, but I don't know where to put the prototype.
                    It's in structs.h now which seems wrong. Feel free to
                    move it to another header.

                    Sumner



                    __________________________________________
                    Yahoo! DSL – Something to write home about.
                    Just $16.99/mo. or less.
                    dsl.yahoo.com
                  • Bram Moolenaar
                    ... Have a look at how items are deep-copied in eval.c Search for copyID. I don t know how Python does this, but I would not be surprised if it works
                    Message 9 of 21 , Jan 10, 2006
                    • 0 Attachment
                      G. Sumner Hayes wrote:

                      > Bram Moolenaar <Bram@...> wrote:
                      > > G. Sumner Hayes wrote:
                      > > Well, you could try doing something complicated and
                      > > see if Vim crashes.
                      >
                      > Deep structures seem okay.
                      >
                      > > Recursive references (e.g., a dict containing
                      > > itself) are always great fun.
                      >
                      > These break. I'm tempted to punt on properly handling
                      > recursive objects for now, and just put in a maximum
                      > recursion depth (as Vim script does when you echo such
                      > a structure); a proper fix requires keeping a table of
                      > visited objects.

                      Have a look at how items are deep-copied in eval.c Search for copyID.
                      I don't know how Python does this, but I would not be surprised if it
                      works similarly.

                      > > > Open issues:
                      > > > 1. What should we do if the eval doesn't return a
                      > > > supported type (I return None, presumably a vim
                      > > > exception will happen and I'm not sure that we
                      > > > want to raise a Python exception.)
                      > >
                      > > Using None might be the best. Or transform it to a
                      > > string with tv2string().
                      >
                      > I don't like a string, since it makes it harder for
                      > Python extensions to test whether the call failed. An
                      > exception might be reasonable, but None is also
                      > reasonable and already implemented so unless someone
                      > has a good argument for the exception I'm inclined to
                      > leave as-is.

                      OK. This also keeps future extensions open.

                      --
                      The 50-50-90 rule: Anytime you have a 50-50 chance of getting
                      something right, there's a 90% probability you'll get it wrong.

                      /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                      /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                      \\\ download, build and distribute -- http://www.A-A-P.org ///
                      \\\ help me help AIDS victims -- http://www.ICCF.nl ///
                    • G. Sumner Hayes
                      ... similarly. Hmm. I m not 100% clear on how it works, before I read through it can you verify that it ll properly handle mutually recursive structures? e.g.
                      Message 10 of 21 , Jan 10, 2006
                      • 0 Attachment
                        Bram Moolenaar <Bram@...> wrote:
                        > G. Sumner Hayes wrote:
                        > > These break. I'm tempted to punt on properly
                        > > handling recursive objects for now, and just
                        > > put in a maximum recursion depth (as Vim script
                        > > does when you echo such a structure); a proper
                        > > fix requires keeping a table of visited objects.
                        >
                        > Have a look at how items are deep-copied in eval.c
                        > Search for copyID. I don't know how Python does
                        > this, but I would not be surprised if it works
                        similarly.

                        Hmm. I'm not 100% clear on how it works, before I
                        read through it can you verify that it'll properly
                        handle mutually recursive structures?

                        e.g.
                        b={}
                        a=[b]
                        b[0]=a

                        and then have deepcopy work okay?

                        Anyway, I did the mapping solution already. Attached.

                        Basically, whenever I'm calling VimToPython on a tv, I
                        store a mapping from the tv to the Python object.
                        Then if I get called on something I've already
                        converted, I just get the Python object out of the
                        mapping instead of building a new one.

                        Tested on mutually recursive objects, works for me.

                        Sumner

                        __________________________________________________
                        Do You Yahoo!?
                        Tired of spam? Yahoo! Mail has the best spam protection around
                        http://mail.yahoo.com
                      • G. Sumner Hayes
                        So after further code review/testing I ve found a reference counting problem in my code that could leak memory. Attached should fix. I _think_ it s
                        Message 11 of 21 , Jan 10, 2006
                        • 0 Attachment
                          So after further code review/testing I've found a
                          reference counting problem in my code that could leak
                          memory. Attached should fix.

                          I _think_ it's feature-complete now that recursive
                          structures are handled, and I've done a fair amount of
                          testing on it so hopefully it's ready for
                          beta-testing.

                          Certainly seems sufficient for getting the output of
                          taglist(), which is the only reason I wrote it in the
                          first place. :-)

                          Sumner

                          __________________________________________________
                          Do You Yahoo!?
                          Tired of spam? Yahoo! Mail has the best spam protection around
                          http://mail.yahoo.com
                        • Bram Moolenaar
                          ... That should work. I tested it a long time ago. ... Good to see you found a solution. It s similar to what happens for deepcopy in eval.c, except that
                          Message 12 of 21 , Jan 11, 2006
                          • 0 Attachment
                            G. Sumner Hayes wrote:

                            > Bram Moolenaar <Bram@...> wrote:
                            > > G. Sumner Hayes wrote:
                            > > > These break. I'm tempted to punt on properly
                            > > > handling recursive objects for now, and just
                            > > > put in a maximum recursion depth (as Vim script
                            > > > does when you echo such a structure); a proper
                            > > > fix requires keeping a table of visited objects.
                            > >
                            > > Have a look at how items are deep-copied in eval.c
                            > > Search for copyID. I don't know how Python does
                            > > this, but I would not be surprised if it works
                            > similarly.
                            >
                            > Hmm. I'm not 100% clear on how it works, before I
                            > read through it can you verify that it'll properly
                            > handle mutually recursive structures?
                            >
                            > e.g.
                            > b={}
                            > a=[b]
                            > b[0]=a
                            >
                            > and then have deepcopy work okay?

                            That should work. I tested it a long time ago.

                            > Anyway, I did the mapping solution already. Attached.
                            >
                            > Basically, whenever I'm calling VimToPython on a tv, I
                            > store a mapping from the tv to the Python object.
                            > Then if I get called on something I've already
                            > converted, I just get the Python object out of the
                            > mapping instead of building a new one.
                            >
                            > Tested on mutually recursive objects, works for me.

                            Good to see you found a solution. It's similar to what happens for
                            deepcopy in eval.c, except that there it stores the info with the copied
                            list or dict (using lv_copyID and lv_copylist or dv_copydict).

                            --
                            hundred-and-one symptoms of being an internet addict:
                            40. You tell the cab driver you live at
                            http://123.elm.street/house/bluetrim.html
                            41. You actually try that 123.elm.street address.

                            /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                            /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                            \\\ download, build and distribute -- http://www.A-A-P.org ///
                            \\\ help me help AIDS victims -- http://www.ICCF.nl ///
                          • G. Sumner Hayes
                            This patch allows Python extensions to call vim.eval() on vim commands that return dict or list structures. ... I ve done a bit more testing and I think it s
                            Message 13 of 21 , Jan 12, 2006
                            • 0 Attachment
                              This patch allows Python extensions to call vim.eval()
                              on vim commands that return dict or list structures.
                              So you can do things like:
                              :py tagmatches = vim.eval("taglist('stat')")

                              I've done a bit more testing and I think it's ready
                              for beta-test/inclusion. Please let me know on
                              vim-dev if there are any problems.

                              Bram, this is the same as the final version I sent in
                              the development thread (the one that fixed the last
                              memory leak I know about).

                              Thanks for your time,

                              Sumner

                              __________________________________________________
                              Do You Yahoo!?
                              Tired of spam? Yahoo! Mail has the best spam protection around
                              http://mail.yahoo.com
                            • Bram Moolenaar
                              ... Thanks. I hope send out a snapshot with updated spell checking tonight, I ll include a few patches tomorrow. -- Q: Should I clean my house or work on Vim?
                              Message 14 of 21 , Jan 12, 2006
                              • 0 Attachment
                                G. Sumner Hayes wrote:

                                > This patch allows Python extensions to call vim.eval()
                                > on vim commands that return dict or list structures.
                                > So you can do things like:
                                > :py tagmatches = vim.eval("taglist('stat')")
                                >
                                > I've done a bit more testing and I think it's ready
                                > for beta-test/inclusion. Please let me know on
                                > vim-dev if there are any problems.
                                >
                                > Bram, this is the same as the final version I sent in
                                > the development thread (the one that fixed the last
                                > memory leak I know about).

                                Thanks. I hope send out a snapshot with updated spell checking tonight,
                                I'll include a few patches tomorrow.

                                --
                                Q: Should I clean my house or work on Vim?
                                A: Whatever contains more bugs.

                                /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net \\\
                                /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
                                \\\ download, build and distribute -- http://www.A-A-P.org ///
                                \\\ help me help AIDS victims -- http://www.ICCF.nl ///
                              • G. Sumner Hayes
                                ... Cool, I m not trying to rush you just making sure it s clear what s ready for inclusion. I ll mark em with the BETA tag when I think they re
                                Message 15 of 21 , Jan 12, 2006
                                • 0 Attachment
                                  Bram Moolenaar <Bram@...> wrote:
                                  > Thanks. I hope send out a snapshot with updated
                                  > spell checking tonight,
                                  > I'll include a few patches tomorrow.

                                  Cool, I'm not trying to rush you just making sure it's
                                  clear what's ready for inclusion. I'll mark 'em with
                                  the BETA tag when I think they're feature-complete and
                                  I've finished up my testing.

                                  Sumner

                                  __________________________________________________
                                  Do You Yahoo!?
                                  Tired of spam? Yahoo! Mail has the best spam protection around
                                  http://mail.yahoo.com
                                • G. Sumner Hayes
                                  ... Is there a public list of features that you re thinking of implementing for Vim 7? Something I could look at and see if anything looks interesting that I
                                  Message 16 of 21 , Jan 12, 2006
                                  • 0 Attachment
                                    Bram Moolenaar <Bram@...> wrote:
                                    > Thanks. I hope send out a snapshot with updated
                                    > spell checking tonight,
                                    > I'll include a few patches tomorrow.
                                    >

                                    Is there a public list of features that you're
                                    thinking of implementing for Vim 7? Something I could
                                    look at and see if anything looks interesting that I
                                    could run with to help you out.

                                    Thanks,

                                    Sumner

                                    __________________________________________________
                                    Do You Yahoo!?
                                    Tired of spam? Yahoo! Mail has the best spam protection around
                                    http://mail.yahoo.com
                                  • Benji Fisher
                                    ... See http://www.vim.org/sponsor/vote_results.php and ... HTH --Benji Fisher
                                    Message 17 of 21 , Jan 12, 2006
                                    • 0 Attachment
                                      On Thu, Jan 12, 2006 at 02:27:40PM -0800, G. Sumner Hayes wrote:
                                      >
                                      > Is there a public list of features that you're
                                      > thinking of implementing for Vim 7? Something I could
                                      > look at and see if anything looks interesting that I
                                      > could run with to help you out.
                                      >
                                      > Thanks,
                                      >
                                      > Sumner

                                      See http://www.vim.org/sponsor/vote_results.php and

                                      :help todo.txt

                                      HTH --Benji Fisher
                                    Your message has been successfully submitted and would be delivered to recipients shortly.