Loading ...
Sorry, an error occurred while loading the content.

Re: [python-iter] More thoughts on Part 2

Expand Messages
  • Guido van Rossum
    ... Hm, that s a silly argument. How many loops (I guess you mean iterations) do you expect in for val in [ x , y , x ] ? Three of course! ... Not any
    Message 1 of 27 , Feb 22, 2001
    • 0 Attachment
      > 1) I think that "for :val in dict" is sematically ambiguous
      > and error prone. Given {1:'x', 2:'y', 3:'x'}, would you
      > get three loops or two?

      Hm, that's a silly argument. How many "loops" (I guess you mean
      iterations) do you expect in "for val in ['x', 'y', 'x']"? Three of
      course!

      > 2) The half colon combinations like "for :val in dict" and
      > "for key: in dict" look unclean to me.

      Not any uncleaner than the very Pythonic list[:hi] and list[hi:], I
      hope? ("Unclean" is a strange word to use in this context anyway.)

      > 3) I think it is useful to think of this project as extending
      > the "for" statement rather than the "in" keyword. This way,
      > we avoid impacting the "if" statement where iteration is
      > not an issue.

      I think of it this way (extending "for"), but lots of people really do
      closely associate "if x in A" and "for x in A", and I can see their
      point: the first is true if the given x is among the values that x is
      set to in the second.

      > 4) GvR's proposed iterator interface seems promising to me.
      > It has the most flexibility, but would be more complex than
      > PEP234's __iter__ method coupled with a loop-counter
      > extension to the "for" statement.

      I think these are pretty close. I actually like the interface in the
      PEP a little better, because it has a single operation that gets the
      next element, rather than a test and a get. I would propose to use
      iterator.next() rather than iterator() though, for reasons explained
      in my previous post.

      > Only minimal modifications are necessary to implement
      > "for key in dict", to expose the for-loop counter,
      > and create __iter__. Each of these can run fast as light.

      I think I agree, though I'm not sure about what you mean by all these.

      > I'm worried that returning returning callable iterator
      > objects will be harder to implement and as slow as an
      > unladen sparrow.

      Are you referring to the proposal to ket dict.keys be the iterator and
      dict.keys() the list? I don't think that has to be slow at all.

      > 5) We should keep an eye on the map() and filter() while
      > we're at this. I would like the following to work:
      > capitalKeyList = map( string.upper, dict )
      > validedKeyList = filter( isvalid, dict )

      Good point. If we implement iterators for dictionaries, it's almost
      inevitable that map() and filter() will iterate over the keys too...

      --Guido van Rossum (home page: http://www.python.org/~guido/)
    • qrczak@knm.org.pl
      ... Obviously three, no matter if it iterated over keys or values (both were proposed). There are three iterations over [ x , y , x ]. But I think we may say
      Message 2 of 27 , Feb 22, 2001
      • 0 Attachment
        Thu, 22 Feb 2001 19:53:48 -0000, othello@... <othello@...> pisze:

        > 1) I think that "for :val in dict" is sematically ambiguous
        > and error prone. Given {1:'x', 2:'y', 3:'x'}, would you
        > get three loops or two?

        Obviously three, no matter if it iterated over keys or values (both
        were proposed). There are three iterations over ['x', 'y', 'x'].

        But I think we may say good-bye to colon proposals and instead think
        about functions and methods producing appropriate sequences, including
        sequences of key,value pairs.

        > 5) We should keep an eye on the map() and filter() while
        > we're at this. I would like the following to work:
        > capitalKeyList = map( string.upper, dict )
        > validedKeyList = filter( isvalid, dict )

        This is problematic, because it would need to construct the container
        "of the same kind as the argument". In Python there is no such concept
        in general. It could be made working for builtin types, but it will not
        work for instances unless a new protocol for that is designed (call
        obj.__class__() and append? what about immutable containers? invent
        obj.__class__.from_sequence(s) after class methods are introduced?).

        What could easily work is:
        capitalKeyList = dict.map (string.upper)
        validedKeyList = dict.filter (isvalid)

        --
        __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
        \__/
        ^^ SYGNATURA ZASTĘPCZA
        QRCZAK
      • Guido van Rossum
        ... Agreed. ... Not needed. map() always returns a list, filter() returns a list unless the argument is a tuple or string. --Guido van Rossum (home page:
        Message 3 of 27 , Feb 22, 2001
        • 0 Attachment
          > But I think we may say good-bye to colon proposals and instead think
          > about functions and methods producing appropriate sequences, including
          > sequences of key,value pairs.

          Agreed.

          > > 5) We should keep an eye on the map() and filter() while
          > > we're at this. I would like the following to work:
          > > capitalKeyList = map( string.upper, dict )
          > > validedKeyList = filter( isvalid, dict )
          >
          > This is problematic, because it would need to construct the container
          > "of the same kind as the argument".

          Not needed. map() always returns a list, filter() returns a list
          unless the argument is a tuple or string.

          --Guido van Rossum (home page: http://www.python.org/~guido/)
        • Magnus Lie Hetland
          ... Hm. I guess this is probably something I ll have to get used to in the end... Does that mean that list(dict) will be equivalent to dict.keys()? I can see
          Message 4 of 27 , Feb 22, 2001
          • 0 Attachment
            On Thu, 22 Feb 2001, Guido van Rossum wrote:

            > > This is problematic, because it would need to construct the container
            > > "of the same kind as the argument".
            >
            > Not needed. map() always returns a list, filter() returns a list
            > unless the argument is a tuple or string.

            Hm. I guess this is probably something I'll have to get used
            to in the end...

            Does that mean that list(dict) will be equivalent to dict.keys()?

            I can see the practicality of the "key in dict" interpretation
            for if statements, but I can't quite see why it must be
            that way for iteration (except possibly to be consistent with
            the if statements). Why not simply have:

            for key, val in dict:
            pass

            as in "for key, val in dict.keys()"? You don't have to use
            the value. And if the overhead of the value is too high,
            perhaps we could add some special syntax just for this
            case (I'm getting dangerously close to the colon-proposal
            here... ;)

            for key, * in dict:
            pass

            And having the same interpretation wherever dict is
            used as a sequence (disregarding
            the if-in thing) woldn't be too bad either, IMO.

            Granted, map/filter would be a bit less useful in this area,
            but with list comprehension that isn't really a big problem,
            in my opinion. For instance:

            bigkeys = [key for (key, val) in dict if key > 10]

            or

            bigvals = [val for (key, val) in dict if val > 10]

            In sum, my position is that list(dict) should be equal
            to dict.items() and that dict.keys() and dict.values()
            should be the special cases. If this is an impossible
            dream, then I'll stop nagging you all about it :)

            --

            Magnus Lie Hetland http://www.hetland.org

            "Reality is what refuses to disappear when you stop
            believing in it" -- Philip K. Dick
          • othello@javanet.com
            ... Three is obvious to me too! I was concerned that out of 100 users, more than a few would answer two because they expected set like behavior from their
            Message 5 of 27 , Feb 22, 2001
            • 0 Attachment
              --- In python-iter@y..., qrczak@k... wrote:
              > Thu, 22 Feb 2001 19:53:48 -0000, othello@j... <othello@j...> pisze:
              >
              > > 1) I think that "for :val in dict" is sematically ambiguous
              > > and error prone. Given {1:'x', 2:'y', 3:'x'}, would you
              > > get three loops or two?
              >
              > Obviously three, no matter if it iterated over keys or values (both
              > were proposed). There are three iterations over ['x', 'y', 'x'].

              Three is obvious to me too! I was concerned that out of 100
              users, more than a few would answer two because they expected
              set like behavior from their values, but I'm only speculating here.

              >
              > But I think we may say good-bye to colon proposals and instead think
              > about functions and methods producing appropriate sequences,
              including
              > sequences of key,value pairs.

              Here, here!

              >
              > > 5) We should keep an eye on the map() and filter() while
              > > we're at this. I would like the following to work:
              > > capitalKeyList = map( string.upper, dict )
              > > validedKeyList = filter( isvalid, dict )
              >
              > This is problematic, because it would need to construct the
              > container "of the same kind as the argument".

              When I first considered maps and filters, I went down the
              same road (return the same type you are given), but bumped
              into a contradiction: "map( lambda c: c, aString )" returns
              a list of characters not a string. That's because map() ALWAYS
              returns a LIST that is the result of APPLYING A FUNCTION to
              each element in the containter.

              Given that map() always returns a list, then filter() must do
              the same so that it maintains the invariant:
              filter( lambda n:1, container ) == map( lambda s=s, container)

              The unifying alternative is to treat all uses of the IN operator,
              maps, or filters, as acting on a container coerced to a list.
              lists coerced to themselves
              string coerced to list(string)
              dicts coerced to dict.keys()
              objs coerced to anything returned by __iter__

              Originally, I thought filter( pred, dict ) should return a
              possibly smaller dictionary. But how would map have a function
              which returns a dictionary element? Better to perform some
              function on the keys or looked up values of the keys.

              TTFN,

              Raymond
            • qrczak@knm.org.pl
              ... They would have to be implemented using __getattr__. I hope this hack will not be chosen. -- __(
              Message 6 of 27 , Feb 24, 2001
              • 0 Attachment
                Thu, 22 Feb 2001 13:49:21 -0500, Guido van Rossum <guido@...> pisze:

                > There's also the revolutionary idea of making dict.keys, dict.items,
                > dict.values without () return iterators,

                They would have to be implemented using __getattr__. I hope this hack
                will not be chosen.

                --
                __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
                \__/
                ^^ SYGNATURA ZASTĘPCZA
                QRCZAK
              • Guido van Rossum
                ... What s wrong with using __getattr__? For built-in objects, this is as fast as calling a method; for class instances, __getattr__ will only be invoked when
                Message 7 of 27 , Feb 24, 2001
                • 0 Attachment
                  > > There's also the revolutionary idea of making dict.keys, dict.items,
                  > > dict.values without () return iterators,
                  >
                  > They would have to be implemented using __getattr__. I hope this hack
                  > will not be chosen.

                  What's wrong with using __getattr__? For built-in objects, this is as
                  fast as calling a method; for class instances, __getattr__ will only
                  be invoked when an attribute is undefined.

                  --Guido van Rossum (home page: http://www.python.org/~guido/)
                • qrczak@knm.org.pl
                  ... With using - only that it s misleading (looks like an attribute access but creates a new object each time it is invoked). And that dir gives a wrong
                  Message 8 of 27 , Feb 24, 2001
                  • 0 Attachment
                    Sat, 24 Feb 2001 12:09:13 -0500, Guido van Rossum <guido@...> pisze:

                    > What's wrong with using __getattr__?

                    With using - "only" that it's misleading (looks like an attribute
                    access but creates a new object each time it is invoked). And that
                    dir gives a wrong answer.

                    Usually I can assume that
                    use1(obj.foo)
                    use2(obj.foo)
                    is equivalent to
                    foo = obj.foo
                    use1(foo)
                    use2(foo)
                    which is not true in this case. These attributes are not only
                    implemented as methods, but they really behave as methods, not as
                    attributes.

                    With being forced to define - that instead of a straightforward
                    method definitions, I have to obscure the code, write more error-prone
                    manual method dispatch, care about inheritance (redirect __getattr__
                    to a superclass), forget about __doc__ etc.

                    And only because of a silly convention which requires to express a
                    particular method as an attribute?! I am really surprised that you
                    accept *this* iteration protocol hack.

                    --
                    __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
                    \__/
                    ^^ SYGNATURA ZASTĘPCZA
                    QRCZAK
                  • Guido van Rossum
                    ... This is a good argument against this! ... Well, defining a sequence type isn t that straightforward anyway. ... Now, wait a minute. I was just asking for
                    Message 9 of 27 , Feb 24, 2001
                    • 0 Attachment
                      > > What's wrong with using __getattr__?
                      >
                      > With using - "only" that it's misleading (looks like an attribute
                      > access but creates a new object each time it is invoked). And that
                      > dir gives a wrong answer.
                      >
                      > Usually I can assume that
                      > use1(obj.foo)
                      > use2(obj.foo)
                      > is equivalent to
                      > foo = obj.foo
                      > use1(foo)
                      > use2(foo)
                      > which is not true in this case. These attributes are not only
                      > implemented as methods, but they really behave as methods, not as
                      > attributes.

                      This is a good argument against this!

                      > With being forced to define - that instead of a straightforward
                      > method definitions, I have to obscure the code, write more error-prone
                      > manual method dispatch, care about inheritance (redirect __getattr__
                      > to a superclass), forget about __doc__ etc.

                      Well, defining a sequence type isn't that straightforward anyway.

                      > And only because of a silly convention which requires to express a
                      > particular method as an attribute?! I am really surprised that you
                      > accept *this* iteration protocol hack.

                      Now, wait a minute. I was just asking for feedback. Can we lose the
                      attitude?

                      --Guido van Rossum (home page: http://www.python.org/~guido/)
                    • gzeljko
                      From: Guido van Rossum ... Let s me understand that in my words, for..in statement is usable 1. For any random acces iterator, using
                      Message 10 of 27 , Feb 24, 2001
                      • 0 Attachment
                        From: Guido van Rossum <guido@...>
                        > > > What's wrong with using __getattr__?
                        > >
                        > > With using - "only" that it's misleading (looks like an attribute
                        > > access but creates a new object each time it is invoked). And that
                        > > dir gives a wrong answer.
                        > >
                        > > Usually I can assume that
                        > > use1(obj.foo)
                        > > use2(obj.foo)
                        > > is equivalent to
                        > > foo = obj.foo
                        > > use1(foo)
                        > > use2(foo)
                        > > which is not true in this case. These attributes are not only
                        > > implemented as methods, but they really behave as methods, not as
                        > > attributes.
                        >
                        > This is a good argument against this!
                        >

                        Let's me understand that in my words, for..in statement is usable

                        1. For any random acces iterator, using 0,1,2... for acces
                        (not maintaining state)
                        Dictionary Iterator doesn't satisfy that.
                        2. For any on the fly constructed iterator providing __getitem__
                        (usage: just trigered with __getitem__, use and maintain internal
                        state)

                        2. is good if explicit, bad if implicit ?

                        ly-y'rs-gzeljko
                      • Ka-Ping Yee
                        ... Among other things, the author of an iterable class would have do tedious and comparatively slow manual dispatch: class Foo: ... def __getattr__(self,
                        Message 11 of 27 , Feb 24, 2001
                        • 0 Attachment
                          On Sat, 24 Feb 2001, Guido van Rossum wrote:
                          > > > There's also the revolutionary idea of making dict.keys, dict.items,
                          > > > dict.values without () return iterators,
                          > >
                          > > They would have to be implemented using __getattr__. I hope this hack
                          > > will not be chosen.
                          >
                          > What's wrong with using __getattr__? For built-in objects, this is as
                          > fast as calling a method; for class instances, __getattr__ will only
                          > be invoked when an attribute is undefined.

                          Among other things, the author of an iterable class would have do
                          tedious and comparatively slow manual dispatch:

                          class Foo:
                          ...
                          def __getattr__(self, name):
                          if name == "keys":
                          ... make key iterator ...
                          elif name == "values":
                          ... make value iterator ...
                          elif name == "items":
                          ... make (key, value) iterator ...
                          else:
                          raise AttributeError, name

                          instead of

                          class Foo:
                          ...
                          def __iterkeys__(self):
                          ... make key iterator ...

                          def __itervalues__(self):
                          ... make value iterator ...

                          def __iteritems__(self):
                          ... make (key, value) iterator ...



                          -- ?!ng

                          "The only `intuitive' interface is the nipple. After that, it's all learned."
                          -- Bruce Ediger, on user interfaces
                        • n8spam@netscape.net
                          ... think ... including ... I seem to be perennially defending disfavored Python proposals, but here goes. I was quite relieved to see Guido s proposed colon
                          Message 12 of 27 , Mar 2, 2001
                          • 0 Attachment
                            --- In python-iter@y..., Guido van Rossum <guido@p...> wrote:
                            > [Qrczak]
                            > > But I think we may say good-bye to colon proposals and instead
                            think
                            > > about functions and methods producing appropriate sequences,
                            including
                            > > sequences of key,value pairs.
                            >
                            > Agreed.


                            I seem to be perennially defending disfavored Python proposals, but
                            here goes. I was quite relieved to see Guido's proposed colon syntax.
                            It settled the discomfort I felt with the ambiguity of the statement:
                            for something in dict:

                            The downside? Let's see...

                            1. It looks a little strange for those used to current Python syntax.
                            This seems to be the most frequently voiced objection, but isn't
                            very compelling, considering how often similar objections are
                            raised against "print>>" and list comprehensions. The Py-dev
                            team doesn't seem to have any qualms about making a few people
                            squirm every now and then in the name of progress.

                            2. There was a criticism that
                            for thing in dict:
                            maps to
                            for key: in dict:
                            while
                            for thing in list:
                            maps to
                            for :item in list:
                            This inconsistency is simple to clear up -- disallow
                            "for thing in dict:" without a colon. It's ambiguous anyhow, and
                            breaks backwards compatability. Note that this is the behavior
                            specified in the PEP.

                            3. You can't use the same syntax for if statements. So what? For if
                            statements you can use has_key() and friends. There's no
                            performance to be gained by allowing "if thing in dict" -- it
                            only introduces a source of ambiguity.

                            4. Is there any criticism 4? Is there something subtle I've missed?


                            On the plus side:

                            1. The colon syntax is compact. Not a huge concern among Pythoneers,
                            but nice when you can get it.

                            2. The programmer can disregard keys or values without dummy
                            variables. This can boost performance of some mapping objects by
                            allowing them to skip the mapping step when only values are
                            requested and thus don't have to be in key order. Nice, elegant
                            icing on the cake.

                            3. It's unambiguous. There's no confusion over whether the loop is
                            iterating over keys, values, or both. I consider this the *major*
                            advantage of the PEP. I just can't agree that "key" is the logical
                            meaning of "thing" in "thing in dict". There's been plenty of
                            anecdotal evidence from T. Wouters and others that I'm not the
                            only one. I can understand that sometimes a default just has to
                            be chosen, but with the colon syntax we have the opportunity to
                            use a completely unambiguous construction. Why not take it?


                            If we're giving up on key:value, what's the proposed alternative? Why
                            is it better?


                            Cheers,
                            -Nathan

                            (My e-mail address at caltech.edu is n8gray)
                          • Clark C. Evans
                            Hello. I am very new to python and find this discussion interesting as iterators and generic functions were the first thing that I marked as notably absent.
                            Message 13 of 27 , Mar 2, 2001
                            • 0 Attachment
                              Hello. I am very new to python and find this discussion
                              interesting as "iterators and generic functions" were the
                              first thing that I marked as notably absent. I'd like to
                              comment, but I don't really have much of a context so if
                              I blunder badly, please forgive me.

                              On Sat, 3 Mar 2001 n8spam@... wrote:
                              > for thing in dict:
                              > for key: in dict:
                              > for :item in list:
                              >
                              > If we're giving up on key:value, what's the proposed
                              > alternative? Why is it better?

                              This syntax has potential. What if the colon could be
                              used for filtering operations as well? For instance,
                              in these examples, let _ be a wild card.

                              for key:_ in dict: # loops through each key
                              for _:value in dict: # loops through each value
                              for key:value in dict: # loops thorugh each key and value

                              for key:'value' in dict: # loops through keys having value = 'value'

                              I don't particularly like ":x" or "x:" since the former
                              reminds me of SQL binding variables, and the latter
                              reminds me of a subordate clause. However, "_:x" and "x:_"
                              make perfect sense to me.

                              There is a likely problem: _ is a valid identifier, right?
                              Well, could "*:value" and "key:*" be used instead then?

                              Sorry if this is _way_ out in left field...

                              Kind Regards,

                              Clark
                            • Clark C. Evans
                              Another question, have people considerd tuple syntax? I actually find this more readable than colon syntax. for (key,_) in dict: for (_,value) in dict: for
                              Message 14 of 27 , Mar 2, 2001
                              • 0 Attachment
                                Another question, have people considerd "tuple" syntax?
                                I actually find this more readable than colon syntax.

                                for (key,_) in dict:
                                for (_,value) in dict:
                                for (key,value) in dict:

                                Once again, sorry if this is way-off base. ;) Clark
                              • qrczak@knm.org.pl
                                ... for k in dict.keys(): for k,v in dict.items(): but with something more appropriate (using some lazy iteration protocol) instead of keys and items.
                                Message 15 of 27 , Mar 3, 2001
                                • 0 Attachment
                                  Sat, 03 Mar 2001 03:29:22 -0000, n8spam@... <n8spam@...> pisze:

                                  > If we're giving up on key:value, what's the proposed alternative?

                                  for k in dict.keys():
                                  for k,v in dict.items():

                                  but with something more appropriate (using some lazy iteration protocol)
                                  instead of keys and items. Possibilities I see or have seen:

                                  1. xkeys(), xitems().
                                  2. keys, items.
                                  3. keys(), items().


                                  Why 1: Doesn't change existing methods or break any existing code.
                                  Doesn't require any magic to work - easy to understand.
                                  Consistent with xrange() and xreadlines().

                                  Why not 1: Introduces an unnecessary duplication of names only to
                                  enable good performance. A programmer should not have to
                                  worry whether he should use keys or xkeys, because their
                                  meaning is essentially the same.


                                  Why 2: A single attribute can be used as either iterator or list
                                  producer, with backward-compatible syntax for the latter.

                                  Why not 2: Attribute syntax plays the role of a method only because of
                                  backward compatibility. Classes must implement this using
                                  __getattr__ because it's really a stateful method call.
                                  Doesn't scale to places when an argument is needed or to
                                  module-scope functions (e.g. range).

                                  Why 3: A programmer says what he wants and Python cares for the rest.
                                  A generic lazy list framework would enable to unify lists and
                                  iterators anywhere both are needed.

                                  Why not 3: Requires much internal magic to let such lazy list behave
                                  in an almost-compatible way with the old regular list - not
                                  sure if it can be done well at all. Breaks code which
                                  expected type(dict.keys()) == types.ListType.

                                  > Why is it better?

                                  Doesn't need separate "iterate over dictionary" and "iterate over
                                  sequence" concepts, but splits the conceptually different things:
                                  - deciding over which sequence to iterate,
                                  - iteration.

                                  --
                                  __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
                                  \__/
                                  ^^ SYGNATURA ZASTĘPCZA
                                  QRCZAK
                                • gzeljko
                                  From: ... Requires diferent evaluation of dict.keys() in diferent context. Maybe it was motivation for colon sintax, reserved for
                                  Message 16 of 27 , Mar 3, 2001
                                  • 0 Attachment
                                    From: <qrczak@...>
                                    >
                                    > Why not 3: Requires much internal magic to let such lazy list behave
                                    > in an almost-compatible way with the old regular list - not
                                    > sure if it can be done well at all. Breaks code which
                                    > expected type(dict.keys()) == types.ListType.
                                    >

                                    Requires diferent evaluation of dict.keys() in diferent context.
                                    Maybe it was motivation for colon sintax, reserved for for-loops.

                                    ly-y'rs-gzeljko
                                  • qrczak@knm.org.pl
                                    ... It does not. It would always be a lazy list, which provides both a sequence protocol and an iteration protocol. The iteration protocol could just reuse
                                    Message 17 of 27 , Mar 3, 2001
                                    • 0 Attachment
                                      Sat, 3 Mar 2001 11:23:31 +0100, gzeljko <gzeljko@...> pisze:

                                      > > Why not 3: Requires much internal magic to let such lazy list behave
                                      > > in an almost-compatible way with the old regular list - not
                                      > > sure if it can be done well at all. Breaks code which
                                      > > expected type(dict.keys()) == types.ListType.
                                      >
                                      > Requires diferent evaluation of dict.keys() in diferent context.

                                      It does not. It would always be a lazy list, which provides both
                                      a sequence protocol and an iteration protocol.

                                      The iteration protocol could just reuse existing slice interface.
                                      A lazy list has fast seq[0] and seq[1:] calls which produce
                                      elements on demand.

                                      Unfortunately the slice interface is slow for normal lists, so it's not
                                      clear how a for loop could know which is better to use. In all cases
                                      both will work, but sometimes one is cheaper, and sometimes the other.

                                      Major problem is the fact that dicts are mutable. In order to preserve
                                      return-by-value semantics of keys(), any modification of the dict
                                      would have to be catched and trigger sucking the remaining keys by
                                      any alive object returned by keys() on that dict. I don't see any big
                                      problems besides this.

                                      To say it more concretely, 'for x in seq: statement' would have the
                                      following semantics for sequences which should better be iterated
                                      over using the slice protocol:

                                      _tmp = seq
                                      while 1:
                                      try: x = _tmp[0]
                                      except IndexError: break
                                      statement
                                      _tmp = _tmp[1:]

                                      Implementation of this protocol by various kinds of lazy lists
                                      (range(), readlines(), keys(), items()) is easy if we ignore the
                                      mutability problem.

                                      --
                                      __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
                                      \__/
                                      ^^ SYGNATURA ZASTĘPCZA
                                      QRCZAK
                                    • qrczak@knm.org.pl
                                      ... Here is a backward compatible proposal. The meaning of for x in seq: statement is as follows: try: _tmp = seq.__iterator__() except AttributeError: _tmp
                                      Message 18 of 27 , Mar 3, 2001
                                      • 0 Attachment
                                        3 Mar 2001 11:57:53 GMT, Marcin 'Qrczak' Kowalczyk <qrczak@...> pisze:

                                        > Unfortunately the slice interface is slow for normal lists, so it's not
                                        > clear how a for loop could know which is better to use. In all cases
                                        > both will work, but sometimes one is cheaper, and sometimes the other.

                                        Here is a backward compatible proposal.
                                        The meaning of 'for x in seq: statement' is as follows:

                                        try: _tmp = seq.__iterator__()
                                        except AttributeError: _tmp = indexing_iterator(seq)
                                        # Types with fast x[1:] can just def __iterator__(self): return self
                                        # Types which don't define __iterator__ get the old iteration protocol.
                                        while 1:
                                        try: x = _tmp[0]
                                        except IndexError: break
                                        statement
                                        _tmp = _tmp[1:]

                                        indexing_iterator is a proxy which emulates sliced iteration interface
                                        in terms of indexed iteration interface.

                                        --
                                        __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
                                        \__/
                                        ^^ SYGNATURA ZASTĘPCZA
                                        QRCZAK
                                      • Ka-Ping Yee
                                        ... I can see that this would work, but i don t understand why you prefer while 1: body(iter[0]) iter = iter[1:] to while 1: # when a
                                        Message 19 of 27 , Mar 3, 2001
                                        • 0 Attachment
                                          On 3 Mar 2001 qrczak@... wrote:
                                          > try: _tmp = seq.__iterator__()
                                          > except AttributeError: _tmp = indexing_iterator(seq)
                                          > # Types with fast x[1:] can just def __iterator__(self): return self
                                          > # Types which don't define __iterator__ get the old iteration protocol.
                                          > while 1:
                                          > try: x = _tmp[0]
                                          > except IndexError: break
                                          > statement
                                          > _tmp = _tmp[1:]
                                          >
                                          > indexing_iterator is a proxy which emulates sliced iteration interface
                                          > in terms of indexed iteration interface.

                                          I can see that this would work, but i don't understand why you prefer

                                          while 1:
                                          body(iter[0])
                                          iter = iter[1:]

                                          to

                                          while 1: # when a new-style iterator is available
                                          body(iter())

                                          or

                                          i = 0
                                          while 1: # what happens when we use make_iterator()
                                          body(iter[i])
                                          i = i + 1

                                          as the basic stepping operation (in the above, think of "body"
                                          as the body of the for-loop and "iter" as the iterator object).


                                          -- ?!ng

                                          "The biggest cause of trouble in the world today is that the stupid people
                                          are so sure about things and the intelligent folk are so full of doubts."
                                          -- Bertrand Russell
                                        • Ka-Ping Yee
                                          ... Binding to tuples already has a well-defined meaning. blah = [(1, 2), (3, 4), (5, 6)] for (a, b) in blah: makes perfect sense, analogous to (a, b) = (1, 2)
                                          Message 20 of 27 , Mar 3, 2001
                                          • 0 Attachment
                                            On Sat, 3 Mar 2001, Clark C. Evans wrote:
                                            > Another question, have people considerd "tuple" syntax?
                                            > I actually find this more readable than colon syntax.
                                            >
                                            > for (key,_) in dict:
                                            > for (_,value) in dict:
                                            > for (key,value) in dict:

                                            Binding to tuples already has a well-defined meaning.

                                            blah = [(1, 2), (3, 4), (5, 6)]
                                            for (a, b) in blah:

                                            makes perfect sense, analogous to (a, b) = (1, 2) in Python.

                                            Similarly --

                                            blah = {1: 2, 3: 4, 5: 6}
                                            for key:value in blah:

                                            It wouldn't make sense (even aside from the compatibility issue!)
                                            to talk about tuples in a context where there aren't any.

                                            Hmm, this has been explained before. Perhaps i should add some
                                            stuff to the Rationale section of the PEP recounting these common
                                            suggestions/objections and their rebuttals.


                                            -- ?!ng

                                            "The biggest cause of trouble in the world today is that the stupid people
                                            are so sure about things and the intelligent folk are so full of doubts."
                                            -- Bertrand Russell
                                          • gzeljko
                                            From: ... a=some_dict.keys() # a going to be list In others words, they must implement complete list interface, 1. without own
                                            Message 21 of 27 , Mar 3, 2001
                                            • 0 Attachment
                                              From: <qrczak@...>
                                              > Sat, 3 Mar 2001 11:23:31 +0100, gzeljko <gzeljko@...> pisze:
                                              >
                                              > > > Why not 3: Requires much internal magic to let such lazy list behave
                                              > > > in an almost-compatible way with the old regular list - not
                                              > > > sure if it can be done well at all. Breaks code which
                                              > > > expected type(dict.keys()) == types.ListType.
                                              > >
                                              > > Requires diferent evaluation of dict.keys() in diferent context.
                                              >
                                              > It does not. It would always be a lazy list, which provides both
                                              > a sequence protocol and an iteration protocol.

                                              a=some_dict.keys() # a going to be 'list'

                                              In others words, they 'must' implement complete list interface,

                                              1. without own backing store
                                              2. with own backing store on demand

                                              1. is clear concept, but here is imposible
                                              a[i] = something
                                              # can't be implemented with old meaning

                                              2. is confusing and impractical (IMHO)

                                              if-they-was-tuples-ly-y'rs-gzeljko
                                            • qrczak@knm.org.pl
                                              ... The former uses an already existing interface. The latter is a new interface. The former doesn t mutate the iterator. Because of this a sequence itself can
                                              Message 22 of 27 , Mar 3, 2001
                                              • 0 Attachment
                                                Sat, 3 Mar 2001 04:42:30 -0800 (PST), Ka-Ping Yee <ping@...> pisze:

                                                > I can see that this would work, but i don't understand why you prefer
                                                >
                                                > while 1:
                                                > body(iter[0])
                                                > iter = iter[1:]
                                                >
                                                > to
                                                >
                                                > while 1: # when a new-style iterator is available
                                                > body(iter())

                                                The former uses an already existing interface. The latter is a new
                                                interface.

                                                The former doesn't mutate the iterator. Because of this a sequence
                                                itself can be its own iterator. For types with fast s[0] and s[1:]
                                                s.__iterator__() can just return self instead of producing a new
                                                proxy object. The __iterator__() method is stateless - it could
                                                even be an attribute instead of a method (but it would introduce
                                                a reference cycle).

                                                The former is a functional style; the latter is imperative. Lazy
                                                lists are immutable by nature. It's much simpler to provide a full
                                                immutable sequence protocol than a full mutable sequence protocol.
                                                For iteration generic mutability is not needed anyway.


                                                I realized that unfortunately the list returned by range() is currently
                                                mutable, so it should probably remain mutable for compatibility. In
                                                case we want to apply the lazy list framework to plain range() and
                                                readlines() retaining their mutability, the picture becomes much more
                                                complicated:-(

                                                Under these assumptions only, we can equally well define the iteration
                                                protocol thus, because iterators are now mutable:

                                                try: _tmp = seq.__iterator__()
                                                except AttributeError: _tmp = indexing_iterator(seq)
                                                while 1:
                                                try: x = _tmp[0]
                                                except IndexError: break
                                                statement
                                                del _tmp[:1] # Here is the difference.

                                                (The builtin list case can be optimized by avoiding the protocol
                                                emulation and proceeding as currently.)

                                                Now __iterator__ must always return a new stateful proxy each time
                                                it is called. Here is an untested subset of the implementation of
                                                mutable lazy range (only unary constructor, no negative indices,
                                                many methods skipped):

                                                class range:
                                                def __init__(self, n):
                                                self.items = []
                                                self.from = 0
                                                self.end = n
                                                # The abstract meaning of this object is
                                                # self.items + [self.from, self.from+1, ..., self.end-1]

                                                def __getitem__(self, i):
                                                try: return self.items[i]
                                                except IndexError:
                                                if i < len(self.items) + self.end - self.from:
                                                return self.from - len(self.items)
                                                raise IndexError

                                                def __setitem__(self, i, x):
                                                try: self.items[i] = x
                                                except IndexError:
                                                if i < len(self.items) + self.end - self.from:
                                                # Must materialize the beginning of the numeric part.
                                                while i > len(self.items):
                                                self.items.append(self.from)
                                                self.from += 1
                                                # Invariant about the abstract meaning restored.
                                                self.items.append(x)
                                                self.from += 1
                                                raise IndexError

                                                def __delslice__(self, i, j):
                                                if i >= j: return
                                                if j > len(self.items) + self.end - self.from:
                                                j = len(self.items) + self.end - self.from
                                                if j <= len(self.items):
                                                # The range is completely inside the self.items part.
                                                del self.items[i:j]
                                                elif i <= len(self.items):
                                                # The range spans both self.items and the numeric part.
                                                self.from += j - len(self.items)
                                                del self.items[i:]
                                                else:
                                                # The range is completely inside the numeric part.
                                                # Must materialize the beginning of the numeric part.
                                                while i > len(self.items):
                                                self.items.append(self.from)
                                                self.from += 1
                                                # Invariant about the abstract meaning restored.
                                                self.from += j - i

                                                The immutable range (i.e. xrange) is much simpler.

                                                Generally functional style (i.e. immutable) is simpler :-)

                                                --
                                                __("< Marcin Kowalczyk * qrczak@... http://qrczak.ids.net.pl/
                                                \__/
                                                ^^ SYGNATURA ZASTĘPCZA
                                                QRCZAK
                                              • gzeljko
                                                From: ... You can think about that in your own terms :) so: dict.keys = stateless lazy list dict.keys() = dict.keys.__call__() - to produce
                                                Message 23 of 27 , Mar 3, 2001
                                                • 0 Attachment
                                                  From: <qrczak@...>
                                                  >
                                                  > 1. xkeys(), xitems().
                                                  > 2. keys, items.
                                                  > 3. keys(), items().
                                                  >
                                                  >
                                                  > Why 1: Doesn't change existing methods or break any existing code.
                                                  > Doesn't require any magic to work - easy to understand.
                                                  > Consistent with xrange() and xreadlines().
                                                  >
                                                  > Why not 1: Introduces an unnecessary duplication of names only to
                                                  > enable good performance. A programmer should not have to
                                                  > worry whether he should use keys or xkeys, because their
                                                  > meaning is essentially the same.
                                                  >
                                                  >
                                                  > Why 2: A single attribute can be used as either iterator or list
                                                  > producer, with backward-compatible syntax for the latter.
                                                  >
                                                  > Why not 2: Attribute syntax plays the role of a method only because of
                                                  > backward compatibility. Classes must implement this using
                                                  > __getattr__ because it's really a stateful method call.
                                                  > Doesn't scale to places when an argument is needed or to
                                                  > module-scope functions (e.g. range).
                                                  >

                                                  You can think about that in your own terms :)

                                                  so:

                                                  dict.keys = stateless lazy list
                                                  dict.keys() = dict.keys.__call__() - to produce old fashion list

                                                  __getattr__-don't-needed-ly-y'rs-gzeljko
                                                Your message has been successfully submitted and would be delivered to recipients shortly.