Loading ...
Sorry, an error occurred while loading the content.
 

Reason for closure/method discrepancy?

Expand Messages
  • victor_yurkovsky
    Can anyone justify closure treatment in self? It would seem to be more in the spirit of self to merge closures with methods and perform a purely dynamic
    Message 1 of 7 , Dec 19, 2001
      Can anyone justify closure treatment in self? It would seem to be
      more in the spirit of self to merge closures with methods and perform
      a purely dynamic message lookup, that is search block
      locals/parameters, then follow parent slots (outer closures - and now
      deprecated 'methods'). This would have an interesting side effect of
      erasing the difference between closures, methods and objects - any of
      these could be stuck into a slot and sent at least a 'value' message.

      A method implementation would become therefore just another object
      (with local, parameter and parent slots) which is sent a 'value'
      message by default, but can accept other messages!

      Then it is up to the programmer to decide whether to write
      a 'block' inside a 'method' (an anonymous closure), create a method
      (a named closure) or send messages to object or methods (that are
      also objects) interchangeably.

      As for the non-local return out of closures, is there a really good
      reason for it? Why not return to the outer scope (other than perhaps
      a clumsier syntax)?

      Forgive my possible stupidity - I am stuck in x86 world and haven't
      been able to explore self personally. I am amazed at the clarity of
      reasoning behind self, with this minor exception. Perhaps someone
      can clarify this.
    • Jecel Assumpcao Jr
      Victor, ... At the language definition level (the implementation is an entirely different story) things are pretty close to what you proposed. But methods are
      Message 2 of 7 , Dec 19, 2001
        Victor,

        > Can anyone justify closure treatment in self? It would seem to be
        > more in the spirit of self to merge closures with methods and perform
        > a purely dynamic message lookup, that is search block
        > locals/parameters, then follow parent slots (outer closures - and now
        > deprecated 'methods').

        At the language definition level (the implementation is an entirely
        different story) things are pretty close to what you proposed. But
        methods are considered prototypes which must be cloned in order to get
        a closure.

        At least in theory, a prototype and its clone would be considered the
        same type or class of object. So it would be correct to say that
        methods and closures are already merged.

        To see why method invocation must generate a clone, just imagine a
        recursive method or one used in several parallel threads. The different
        execution contexts would want to store different values in the argument
        slots and local slots, so we can't get by with just one.

        > This would have an interesting side effect of
        > erasing the difference between closures, methods and objects - any of
        > these could be stuck into a slot and sent at least a 'value' message.

        Would fetching something from a slot automatically involve sending
        'value' to what was found there? If so, blocks would no longer work
        properly (more below). If not, methods would no longer work properly.

        > A method implementation would become therefore just another object
        > (with local, parameter and parent slots) which is sent a 'value'
        > message by default, but can accept other messages!

        A closure (context object, activation, whatever name you want) can
        already accept many messages, though the current implementation doesn't
        deal with method slots inside methods.

        > Then it is up to the programmer to decide whether to write
        > a 'block' inside a 'method' (an anonymous closure), create a method
        > (a named closure) or send messages to object or methods (that are
        > also objects) interchangeably.

        Blocks turn out to be much more complicated than they appear at first.
        In fact, we need three separate objects to support them:

        block literal: this bundles up all the information in source and
        gets stored in the method object

        block context: this is the name I gave this in my tinySelf
        interpreter, but it really isn't a context object at all. It must point
        to the block literal for later use, point to the currently executing
        context for later use, and have 'traits block' as a parent so we can
        send all kinds of interesting messages to it ('value' is the most
        important, but we need 'whileTrue:', 'millisecondsToRun' and many
        others)

        block method context: this is created as the result of sending
        'value' to the block context. It is cloned from the method found in the
        block literal but its parent slot is set to point to the context saved
        in the block context instead of the receiver like for normal methods.
        This way we have a chain of closures like you wanted.

        For a single block literal we might create several block contexts since
        the method in which it appears might be invoked multiple times. For a
        single block context we might create seveal block method contexts since
        it might receive several 'value' messages.

        > As for the non-local return out of closures, is there a really good
        > reason for it? Why not return to the outer scope (other than perhaps
        > a clumsier syntax)?

        Unless you have a "^" before the last expression in a block, it does
        return to the outer scope.

        [ 1 + 2. 3 + 4 ] value

        But that is not always what you want. Sometimes you need to "unwind" a
        deeply nested computation:

        ....
        arg < 0 ifTrue: [ ^ 1 ].
        ....

        This lets us relax the rules of strict structured programming.
        Otherwise we would have to do:

        ....
        arg < 0 ifTrue: [ 1 ]
        False: [
        ....]

        For many real programs, being unable to escape inner scopes in an
        unstructured way can become quite painful (see Pascal vs C).

        > Forgive my possible stupidity - I am stuck in x86 world and haven't
        > been able to explore self personally. I am amazed at the clarity of
        > reasoning behind self, with this minor exception. Perhaps someone
        > can clarify this.

        These are very good questions and I hope I understood them correctly
        and was able to explain some of the issues involved. I have been trying
        to simplify blocks or unify them with methods since my 1984 Smalltalk
        design (http://www.lsi.usp.br/~jecel/st84.txt) and would very much like
        to see a simpler solution for Self.

        -- Jecel
      • victor_yurkovsky
        Jecel, thank you for a quick response. Firstly, I have no problem with the cloning of the activation records, etc. That makes total sense. I do have issues
        Message 3 of 7 , Dec 19, 2001
          Jecel, thank you for a quick response.

          Firstly, I have no problem with the cloning of the activation
          records, etc. That makes total sense.
          I do have issues with lexical scoping in interpreted environments
          generally, and especially with pure object-oriented languages. Other
          than the lexical issues, self is a clear gem.

          My problem starts with self curtailing the lifespan of closures to
          the enclosing method. Although it saves the implementation from
          dealing with moving activation records off the stack, it cripples the
          language. I very much like the idea of passing closures around and
          sticking them into slots.

          Also, at the risk of getting off-topic, I need to get to the bottom
          of the closure/method issue. Once again, apologies for not being up
          on self. This is the way I envision the ideal OO language (and self
          is really close...)

          1) There is an environment supporting objects.
          2) Objects contain slots and a piece of code.
          3) Each slot consists of a name and a pointer to another object. The
          name dynamically binds the referenced object and allows code to refer
          to it.
          4) The code is in some manner human-modifiable and executable by the
          machine (ignore implementation details here).
          5) Through code, the environment can modify both the shape of an
          object (number and naming of slots) as well as the contents of the
          slots.
          6) Object visibility: The topology of an object, at run-time, defines
          the visibility of objects other than those referenced by its own
          slots (dynamic scoping of objects). The name-binding process is
          equivalent to a search of named slots, starting at the object in
          question, followed by objects referred to by its slots, etc (somehow
          avoiding loops).

          This seems to roughly describe an object-oriented environment capable
          of supporting a decent language. Now the details of interpretation I
          am not solid on right now, but there are a few ways of doing it:

          - Find the target object by name - search proceeds as in 6) above.
          - Execute its code, returing the target reference.
          - Find the 'message' withing the dynamic scope of the target object.
          If present, evaluate it with whatever parameters.

          Sending a 'no message' message returns the value (equivalent to the
          execution of a method, sending 'value' to a block or fetching a data
          slot). Note that an object can be referred to by more than one name
          if it's bound to more than one slot.

          For example, let's say we have an object foo with slots named A B and
          C. We can assign an integer 4 to A; a method [A * 2] to B; and a
          closure myClosure to C. Now, there is no difference between blocks
          and method, eh? The code is completely decoupled from any bindings
          until runtime.

          This arrangement has several benefits over the smalltalk/self model:
          you can totally redirect objects to proxies, or do anything at all at
          the point of lookup and closures are in fact real objects.

          bar. (returns the result of evaluating an object at a slot
          name bar in self or above).
          bar increment. (returns the result of evaluation of an object
          at the slot named 'increment' found at or above an object at the slot
          named 'bar' in self or above).
          bar := 4 (send := 4 to the object at a slot named 'bar' in
          self or above)...

          Sorry about the rambling, but I've been thinking about this for too
          long the wrong way, and the recent brush with self is beginning to
          clear my head (but it's not quite clear yet...)
        • victor_yurkovsky
          Jecel, thanks for your answers. Let s postpone the activation record discussion (I am very curious about the double-cloning and istalling as own parent
          Message 4 of 7 , Dec 20, 2001
            Jecel, thanks for your answers.

            Let's postpone the activation record discussion (I am very curious
            about the double-cloning and istalling as own parent thought..) as an
            implementation issue.

            Now, the lifespan of a block, as you pointed out, is limited to the
            lexically enclosing method. As an implementer, I can understand the
            desire to keep activation records on the stack for efficiency, and
            that implies that the record cannot survive the stack frame of the
            method. However, it does preclude me from passing a block out of a
            method and sticking it into a slot of a more permanent object. That
            seems like a deficiency as I often find myself (in smalltalk) in
            situations where I really wish to replace a method with another one
            after the state of the object changes (instead of a slower
            conditional). Am I missing something here - is there another
            mechanism in self to replace the contents of a slot with another
            (precompiled or dynamically generated) entity? I guess you can
            replace the contents of a slot with anything you want, so that would
            be the way to do it. It just really yanks me that the creation of a
            block and a method is not syntactically identical.

            As for scoping, do I really understand it correctly that a block
            statically binds to objects of the enclosing method at compile time?
            It would be just so much cleaner to bind at run time... Ah... It
            seems like there are at least two distinct uses of a block:
            partitioning a method (requires lexical access to method slots), the
            other a generic submethod to be passed around (such as a sort block),
            requires dynamic access to run-time method's slots that can be faked
            by passing parameters. Perhaps these should be treated as different
            animals, or the programer be given the choice of lexical/dynamic
            binding of a block? Of course, dynamic binding works in both cases
            as the partition-type block is executed within the same method it was
            compiled in...

            To clarify my seminonsensical last message, it seems that there is a
            more generic way to implement an oo environment within which self can
            be instantiated, just like smalltalk can be instantiated in self...
            The object with slots and data/code (data of course is code
            interpreted by some program or hardware...) seems like about as low
            as I would be willing to go. From there on, it is mostly the
            question of when to bind. Bind everything at compile time and you
            have Forth. Bind objects at compile time and messages at runtime,
            and you have Smalltalk. Bind everything at runtime, and what do you
            have?

            You are totally right about my missing out on the assignment issues.
            The big problem here is name binding again - do names bind the slots
            or objects inside those slots? Is a slot, perhaps, an object capable
            of assignment? I have to think about it some more as I am quickly
            approaching middle age and my mind is not working as well as it did
            (or at least as I remember it doing).
          • Jecel Assumpcao Jr
            ... Ok. One complaint some people have about the cloning is that the methods don t have :self* slots but activation records do. I guess these slots (note
            Message 5 of 7 , Dec 20, 2001
              > Firstly, I have no problem with the cloning of the activation
              > records, etc. That makes total sense.

              Ok. One complaint some people have about the "cloning" is that the
              methods don't have ':self*' slots but activation records do. I guess
              these slots (note that they are argument and parent) could be added to
              methods to make things more uniform while wasting a little space.

              But actually, activation records are continuations and not just
              closures. They have an instruction pointer and a local data stack,
              neither of which was in the original method object.

              In some of my implementations I have dealt with this by slightly
              redefining the "method story". The method is indeed a prototype for the
              closure, but when it is cloned a second object is also cloned and
              installed at its parent:

              ( | :self*.
              <pc> <- 0.
              <stk> <- vector copySize: 16 FillingWith: 0.
              <sp> <- 0.
              | )

              where <name> indicates a slot name that can't be accessed via normal
              message sends. This object plus the cloned method would form an
              activation record.

              > I do have issues with lexical scoping in interpreted environments
              > generally, and especially with pure object-oriented languages. Other
              > than the lexical issues, self is a clear gem.

              I created a Self-like language, NeoLogo, with dynamic scopes. It is
              cleaner, but gets the programmers into trouble more easily.

              > My problem starts with self curtailing the lifespan of closures to
              > the enclosing method. Although it saves the implementation from
              > dealing with moving activation records off the stack, it cripples the
              > language. I very much like the idea of passing closures around and
              > sticking them into slots.

              Here is a quote from "Towards a Universal Implementation Substrate for
              Object-Oriented Languages" by Mario Wolczko, Ole Agesen, David Ungar:

              "Life-time of blocks. Self blocks cannot be invoked after their
              enclosing method returns. Lifting this restriction makes it easier to
              translate Smalltalk blocks that use this feature, as well as
              translating other languages with closure-like constructs."

              > Also, at the risk of getting off-topic, I need to get to the bottom
              > of the closure/method issue. Once again, apologies for not being up
              > on self. This is the way I envision the ideal OO language (and self
              > is really close...)
              >
              > 1) There is an environment supporting objects.
              > 2) Objects contain slots and a piece of code.

              Either one can be missing in a given object, though.

              > 3) Each slot consists of a name and a pointer to another object. The
              > name dynamically binds the referenced object and allows code to refer
              > to it.
              > 4) The code is in some manner human-modifiable and executable by the
              > machine (ignore implementation details here).
              > 5) Through code, the environment can modify both the shape of an
              > object (number and naming of slots) as well as the contents of the
              > slots.

              One major decision is how common this will be. If "normal" code doesn't
              do this, but only the IDE as a result of a direct request by the
              programmer, then it might be acceptable for these modifications to take
              up to a few seconds.

              > 6) Object visibility: The topology of an object, at run-time, defines
              > the visibility of objects other than those referenced by its own
              > slots (dynamic scoping of objects). The name-binding process is
              > equivalent to a search of named slots, starting at the object in
              > question, followed by objects referred to by its slots, etc (somehow
              > avoiding loops).

              You don't want to search all slots, just parents or some equivalent
              notion. Searches through "normal" slots can happen manually, as you
              explained below.

              I would just like to point out that there are other links in the object
              topology that we normally don't think about: the literals inside the
              object's code. I find myself appreciating more and more their
              importance.

              > This seems to roughly describe an object-oriented environment capable
              > of supporting a decent language. Now the details of interpretation I
              > am not solid on right now, but there are a few ways of doing it:
              >
              > - Find the target object by name - search proceeds as in 6) above.

              This would be an implicit self send.

              > - Execute its code, returing the target reference.
              > - Find the 'message' withing the dynamic scope of the target object.
              > If present, evaluate it with whatever parameters.

              This would be a normal send.

              > Sending a 'no message' message returns the value (equivalent to the
              > execution of a method, sending 'value' to a block or fetching a data
              > slot). Note that an object can be referred to by more than one name
              > if it's bound to more than one slot.

              Ok, except for assignment your "story" seems to work as an explanation
              for Self's syntax and semantics.

              > For example, let's say we have an object foo with slots named A B and
              > C. We can assign an integer 4 to A; a method [A * 2] to B; and a
              > closure myClosure to C. Now, there is no difference between blocks
              > and method, eh? The code is completely decoupled from any bindings
              > until runtime.

              It seems to work with your examples, but things would be more
              interesting if you had a method with arguments and/or local slots in B.

              > This arrangement has several benefits over the smalltalk/self model:
              > you can totally redirect objects to proxies, or do anything at all at
              > the point of lookup and closures are in fact real objects.
              >
              > bar. (returns the result of evaluating an object at a slot
              > name bar in self or above).
              > bar increment. (returns the result of evaluation of an object
              > at the slot named 'increment' found at or above an object at the slot
              > named 'bar' in self or above).
              > bar := 4 (send := 4 to the object at a slot named 'bar' in
              > self or above)...

              If you are trying to do assignment, it won't work this way. By the time
              object 'bar' gets the ':= 4' message, you no longer have a reference to
              the object with the 'bar' slot, and that is what you want to change, I
              think.

              > Sorry about the rambling, but I've been thinking about this for too
              > long the wrong way, and the recent brush with self is beginning to
              > clear my head (but it's not quite clear yet...)

              If it was easy, someone would have done it already and we wouldn't have
              things like C# as examples of great advances in language design. I put
              some ideas I have been working on here:

              http://www.merlintec.com:8080/software/

              I probably won't be doing the object format or bytecodes as described
              there. These are just some thought experiments.

              -- Jecel
            • Jecel Assumpcao Jr
              ... The quote in my previous message was meant to imply that the main Self people are unhappy with the situation you describe above and that this is very
              Message 6 of 7 , Dec 20, 2001
                > Now, the lifespan of a block, as you pointed out, is limited to the
                > lexically enclosing method. As an implementer, I can understand the
                > desire to keep activation records on the stack for efficiency, and
                > that implies that the record cannot survive the stack frame of the
                > method. However, it does preclude me from passing a block out of a
                > method and sticking it into a slot of a more permanent object.

                The quote in my previous message was meant to imply that the main Self
                people are unhappy with the situation you describe above and that this
                is very likely to change in the future.

                > That
                > seems like a deficiency as I often find myself (in smalltalk) in
                > situations where I really wish to replace a method with another one
                > after the state of the object changes (instead of a slower
                > conditional). Am I missing something here - is there another
                > mechanism in self to replace the contents of a slot with another
                > (precompiled or dynamically generated) entity? I guess you can
                > replace the contents of a slot with anything you want, so that would
                > be the way to do it. It just really yanks me that the creation of a
                > block and a method is not syntactically identical.

                It is very simple to have many slightly different objects in Self, so
                many design patterns that use non-LIFO blocks in Smalltalk can easily
                be implemented with "anonymous objects" in Self. Smalltalk's

                ....
                sortBlock := [ :a :b | a age > b age ].
                ....
                sortBlock value: arg1 value: arg2
                ....
                sortBlock := [ :a :b | a weight > b weight ].

                can easily be implemented in Self as

                ....
                sortObj: ( | compare: a And: b = ( a age > b age ) | ).
                ....
                sortObj compare: arg1 With: arg2
                ....
                sortObj: ( | compare: a And: b = ( a weight > b weight ) | ).

                It is more awkward in the simple cases, but actually becomes more
                usable than the Smalltalk alternative when you want the parameter
                objects to understand more than one message.

                > As for scoping, do I really understand it correctly that a block
                > statically binds to objects of the enclosing method at compile time?

                No - a clone of the block's value method is bound at run time to a
                clone of the enclosing method. That is a rather different thing from
                saying that the block object is statically bound to the enclosing
                method object at compile time (which is true). See my explanation in
                the previous email about the three different objects needed to
                implement blocks.

                > It would be just so much cleaner to bind at run time... Ah... It
                > seems like there are at least two distinct uses of a block:
                > partitioning a method (requires lexical access to method slots), the
                > other a generic submethod to be passed around (such as a sort block),
                > requires dynamic access to run-time method's slots that can be faked
                > by passing parameters. Perhaps these should be treated as different
                > animals, or the programer be given the choice of lexical/dynamic
                > binding of a block? Of course, dynamic binding works in both cases
                > as the partition-type block is executed within the same method it was
                > compiled in...

                Not if I understood what you meant by "partition-type block". Consider

                .... x < 0 ifTrue: [ x: 0 ].

                This block is actually executed inside the 'ifTrue:' method in the
                boolean object 'true', not the current method within the current
                receiver ('self'). For dynamic scoping to work we need to have some
                link going from the 'ifTrue:' clone (closure) to the current execution
                context (where 'x' is defined) *and* 'ifTrue:' itself must *not* define
                its own 'x' slot. The first problem is also present in lexical scoping
                (and it less elegantly solved there) but at least we get rid of the
                second problem.

                > To clarify my seminonsensical last message, it seems that there is a
                > more generic way to implement an oo environment within which self can
                > be instantiated, just like smalltalk can be instantiated in self...

                That is what I am looking for.

                > The object with slots and data/code (data of course is code
                > interpreted by some program or hardware...) seems like about as low
                > as I would be willing to go. From there on, it is mostly the
                > question of when to bind. Bind everything at compile time and you
                > have Forth. Bind objects at compile time and messages at runtime,
                > and you have Smalltalk. Bind everything at runtime, and what do you
                > have?

                I like Forth :-) You lost me in the part about Smalltalk binding
                objects at compile time.

                > You are totally right about my missing out on the assignment issues.
                > The big problem here is name binding again - do names bind the slots
                > or objects inside those slots? Is a slot, perhaps, an object capable
                > of assignment? I have to think about it some more as I am quickly
                > approaching middle age and my mind is not working as well as it did
                > (or at least as I remember it doing).

                How do you know that you mind doesn't work better now but your memory
                of how it used to be has been damaged? ;-)

                You might enjoy a previous thread I started about assignment and
                arguments on this list:

                http://groups.yahoo.com/group/self-interest/message/1066

                -- Jecel
              • victor_yurkovsky
                Thanks for all the insights. I have to seriously think about it and try out a few things in Mr. Gliebe s port (Thank you!!!) so I don t sound like a complete
                Message 7 of 7 , Dec 20, 2001
                  Thanks for all the insights. I have to seriously think about it and
                  try out a few things in Mr. Gliebe's port (Thank you!!!) so I don't
                  sound like a complete fool (too late).

                  By the way, what I meant by compile-time binding of objects in
                  Smalltalk is that objects that receive messages are either explicitly
                  returned from somewhere (this part is dynamic, of course) or are
                  statically bound to a named instance (or class) variable or a pool.
                  I think I was suggesting an extra object lookup (conceptually anyway
                  as it can be optimized out) if an object is referred to by name,
                  which I realize self in fact does by searching the slot hierarchy.
                  The litmus test for me is whether it's possible at runtime to modify
                  the object's inheritance path and have it function correctly in the
                  new environment (that is, have the messages sent to the newly-bound
                  objects if necessary).

                  Thanks for hammering the cloning thing into me, it does make a
                  difference.

                  What was I talking about anyway? I do suffer from CRS (Can't
                  Remember S**t)
                Your message has been successfully submitted and would be delivered to recipients shortly.