Loading ...
Sorry, an error occurred while loading the content.

Shall I start adding iterators to Python 2.2?

Expand Messages
  • Guido van Rossum
    I ve got a fairly complete implementation of iterators along the lines of Ping s PEP (slightly updated). This is available for inspection through CVS: just
    Message 1 of 3 , Apr 19, 2001
    • 0 Attachment
      I've got a fairly complete implementation of iterators along the lines
      of Ping's PEP (slightly updated). This is available for inspection
      through CVS: just check out the CVS branch of python/dist/src named
      "iter-branch" from SourceForge:

      cvs checkout -r iter-branch -d <directory> python/src/dist

      (This branch was forked off the trunk around the time of version
      2.1b1, so it's not up to date with Python 2.1, but it's good enough to
      show off iterators.)

      My question is: should I just merge this code onto the trunk (making
      it part of 2.2), or should we review the design more before committing
      to this implementation?

      Brief overview of what I've got implemented:

      - There is a new built-in operation, spelled as iter(obj) in Python
      and as PyObject_GetIter(obj) in C; it calls the tp_iter slot in
      obj's type. This returns an iterator, which can be any object that
      implements the iterator protocol. The iterator protocol defines one
      method, next(), which returns the next value or raises the new
      StopIteration exception.

      - For backwards compatibility, if obj's type does not have a valid
      tp_iter slot, iter(obj) and PyObject_GetIter(obj) create a sequence
      iterator that iterates over a sequence.

      - "for x in S: B" is implemented roughly as

      __iter = iter(S)
      while 1:
      try:
      x = __iter.next()
      except StopIteration:
      break
      B

      (except that the semantics of break when there's an else clause are
      not different from what this Python code would do).

      - The test "key in dict" is implemented as "dict.has_key(key)". (This
      was done by implementing the sq_contains slot.

      - iter(dict) returns an iterator that iterates over the keys of dict
      without creating a list of keys first. This means that "for key in
      dict" has the same effect as "for key in dict.keys()" as long as the
      loop body doesn't modify the dictionary (assignment to existing keys
      is okay).

      - There's an operation to create an iterator from a function and a
      sentinel value. This is spelled as iter(function, sentinel). For
      example,

      for line in iter(sys.stdin.readline, ""):
      ...

      is an efficient loop over the lines of stdin.

      - But even cooler is this, which is totally equivalent:

      for line in sys.stdin:
      ...

      - Not yet implemented, but part of the plan, is to use iterators for
      all other implicit loops, like map/reduce/filter, min/max, and the
      "in" test for sequences that don't define sq_contains.

      --Guido van Rossum (home page: http://www.python.org/~guido/)
    • Tim Peters
      [Guido] ... My answer is both! *Most* of what you described is no longer controversial; 2.2 is mondo pre-alpha (so we re not stuck with anything you check
      Message 2 of 3 , Apr 20, 2001
      • 0 Attachment
        [Guido]
        > I've got a fairly complete implementation of iterators along the lines
        > of Ping's PEP (slightly updated).
        > ...
        > My question is: should I just merge this code onto the trunk (making
        > it part of 2.2), or should we review the design more before committing
        > to this implementation?

        My answer is both! *Most* of what you described is no longer controversial;
        2.2 is mondo pre-alpha (so we're not "stuck" with anything you check in now);
        and it's much more convenient (for me - heh) to try out if it's in the
        regular build tree. I bet Greg Wilson would like it for his Set PEP work
        too, as abusing the __getitem__ protocol for set iteration is giving him
        headaches. WRT what may still be controversial points, there's no substitute
        for trying a thing.

        > ...
        > - The test "key in dict" is implemented as "dict.has_key(key)". (This
        > was done by implementing the sq_contains slot.

        That's probably controversial, but also easy to rip out (sounds approximately
        self-contained) if the peasants storm your castle with flaming dungballs
        <wink>.

        > - iter(dict) returns an iterator that iterates over the keys of dict
        > without creating a list of keys first. This means that "for key in
        > dict" has the same effect as "for key in dict.keys()" as long as
        > the loop body doesn't modify the dictionary (assignment to existing
        > keys is okay).

        Ditto.

        > - There's an operation to create an iterator from a function and a
        > sentinel value. This is spelled as iter(function, sentinel). For
        > example,
        >
        > for line in iter(sys.stdin.readline, ""):
        > ...
        >
        > is an efficient loop over the lines of stdin.
        >
        > - But even cooler is this, which is totally equivalent:
        >
        > for line in sys.stdin:
        > ...

        Here you're going to be hoisted on your own petard (Jeremy can explain that
        one <wink>): if

        for x in dict:

        has to iterate over keys because

        if x in dict:

        tests for keys, then shouldn't

        if line in sys.stdin:

        also check sys.stdin for the presence of line? Not according to me, but it's
        a not wholly unreasonable question.

        > - Not yet implemented, but part of the plan, is to use iterators for
        > all other implicit loops, like map/reduce/filter, min/max, and the
        > "in" test for sequences that don't define sq_contains.

        Check it into the trunk and people can help out with that stuff.

        +0.9.
      • Thomas Wouters
        ... I don t totally agree. Removing something from the trunk is not as easy as not adding it ;) But I agree that, since the *concept* of iterators, and the
        Message 3 of 3 , Apr 20, 2001
        • 0 Attachment
          On Fri, Apr 20, 2001 at 03:15:30AM -0400, Tim Peters wrote:
          > [Guido]
          > > I've got a fairly complete implementation of iterators along the lines
          > > of Ping's PEP (slightly updated).
          > > ...
          > > My question is: should I just merge this code onto the trunk (making
          > > it part of 2.2), or should we review the design more before committing
          > > to this implementation?

          > My answer is both! *Most* of what you described is no longer controversial;
          > 2.2 is mondo pre-alpha (so we're not "stuck" with anything you check in now);
          > and it's much more convenient (for me - heh) to try out if it's in the
          > regular build tree. I bet Greg Wilson would like it for his Set PEP work
          > too, as abusing the __getitem__ protocol for set iteration is giving him
          > headaches. WRT what may still be controversial points, there's no substitute
          > for trying a thing.

          I don't totally agree. Removing something from the trunk is not as easy as
          not adding it ;) But I agree that, since the *concept* of iterators, and the
          basic implementation, all are good things, they should be checked in. I
          still don't like:

          > > ...
          > > - The test "key in dict" is implemented as "dict.has_key(key)". (This
          > > was done by implementing the sq_contains slot.

          > That's probably controversial, but also easy to rip out (sounds approximately
          > self-contained) if the peasants storm your castle with flaming dungballs
          > <wink>.

          Fetchez-la-vache!-ly y'rs

          (Oh, now I get it... Iterators are Guido's wooden rabbit, with 'key-in-dict'
          hidden inside... I just hope it's Sir Bedevere that's building it ;)

          --
          Thomas Wouters <thomas@...>

          Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
        Your message has been successfully submitted and would be delivered to recipients shortly.