Python Iterators and Generators

__iter__and__next__

Let’s start with a common use of a for loop in development:

A =[1,2,3] for I in a: print(I) 1 2 3

A is a list and is itself an iterator; Must have the iterator for loop to iterate through the, will default in traversed call iterator iter () method, it displayed, which is equivalent to:

for i in iter(a):
    print(i)
1
2
3

Before we can understand iterators, we must understand the calls to the magic methods __iter__ and __next__. Behind iter(a) is a call to the __iter__() method of object A, i. e. a.__iter__();

What does iter(a) return? Returns an object that implements the __next__() method internally, and completes the traversal each time it calls __next__() to popup the next value.

For example:

class A(object): def __init__(self,a): self.a=a def __next__(self): print('this is next') if self.a<6: self.a+=1 return self.a else: raise StopIteration class B(object): def __init__(self): pass def __iter__(self): Print (' this is iter) return A (A = 0) > > > b = b () > > > for I in b: # is essentially calls iter (b), or b. __iter__ (); Returns an instance object that implements the __next__ method. >>> print(I) this is iter# iter(b) this is next# next 1 this is next 2 this is next 3 this is next 4 this is next 5 this is next 6 this is next #

It is worth noting two concepts:

  • Iterable objects: Implemented in class__iter__I would call it iterable__iter__What is returned is an implementation__next__Object instance of the. This is not mandatory. If it is realized by itself__next__Method, which naturally returns self.
  • Iterator: Implemented in class__iter__and__next__Object of the method. You can’t call it an Iterator without any of them; An iterator is also, of course, an Iterable.

A(0) is neither Iterable nor Iterator. A(0) is neither Iterable nor Iterator. B() is Iterable but not Iterator

from collections.abc import Iterable,Iterator
>>>isinstance(a, Iterable)
False
>>>isinstance(a, Iterator)
False
>>>isinstance(b, Iterable)
True
>>>isinstance(b, Iterator)
False

Python’s built-in iterable objects: lists, tuples, dictionaries, collection strings, open() files, and so on; Notice that these are not iterators; Because we can’t know the length of the sequence ahead of time with iterators, we have to constantly evaluate the next data on demand through the next() function, so the Iterator is lazy and only evaluates when the next data needs to be returned. And obviously lists, tuples, dictionaries, etc., we can have global information.

The Iterator Iterator

Define our iterators based on __iter__ and __next__ :

class Myrange(object):
    def __init__(self,stop,start=0):
        self.start=start
        self.stop=stop
    def __iter__(self):
        return self
    def __next__(self):
        if self.start<self.stop:
            res=self.start
            self.start+=1
        else:
            raise StopIteration
        return res
    
for i in Myrange(3):
    print(i)   
0
1
2

>>>isinstance(Myrange(3),Iterable)
True
>>>isinstance(Myrange(3),Iterator)
True
>>>type(Myrange(3))
<class '__main__.Myrange'>

The generator

Python provides a special yield method that converts a function object directly to an iterator. Generators are a special class of iterators. Whenever a program executes at yield, it pauses there, suspends, and then returns whatever is returned before resuming execution.

def func(end): start=0 while start<end: yield start start+=1 from collections.abc import Iterable,Iterator >>>print(isinstance(func(3),Iterator)) >>>print(isinstance(func(3),Iterable)) >>>for i in func(3): > > > print (I) True# is a iterative True# is an iterator object 0 1 2 > > > a = func (4) > > > print (next) (a) > > > print (a. __next__ ()) 0 1 > > > print (type (a)) <class 'generator'> >>>print(isinstance(a, generator))# is also a generator type. <class 'generator'>

As you can see from the principle of iterators or generators, one of their biggest features is memory savings. Similarly, traversing a list of 100 million integers and a generator consumes completely different amounts of memory. The list needs to be completely cleared out of physical space to save the data first, and the generator needs to save only one data.

import sys
>>>a=range(1000000)
>>>sys.getsizeof(a)
48
>>>b=list(range(1000000))
>>>sys.getsizeof(b)
8000056

The scenario used by the generator has the benefit of saving memory, provided that we can calculate by some algorithm, in the process of the loop constantly calculate the subsequent desired elements.

conclusion

  • To achieve the__iter__The object of the method is an Iterable object. Common Iterable objects include list,dict,tuple,range and so on. At the same time__iter__and__next__Method is an Iterator. The object in a function where the yield keyword is applied is the Generator, which is also an iterator and an iterable object.
  • Both the Iterator and the Generator need to either go through the for loop or manually call next() to get their internal values one by one
  • The use of iterators and generators can save memory, but only if we are able to use some algorithm to calculate the desired elements as we go through the loop.
  • Other usage scenario generator can refer to https://www.zhihu.com/questio…
  • Finally, the relationship between containers, iterators, generators, and iterable objects can be summarized as follows:

在这里插入图片描述