Python3.

1. Python magic

Methods wrapped in the double underscore __ in Python are known as magic methods, which can be used to provide classes with arithmetic, logical operations, and so on, allowing them to perform these operations in a more standard and concise manner than native objects. Here are a few magic tricks that are often asked.

1.1 __init__

What the __init__ method does is initialize the variable after the object is created. Many people think __init__ is a constructor, but it’s not. The __new__ method that actually creates an instance is the one that we’ll look at next.

class Person(object):
    def __init__(self, name, age):
        print("in __init__")
        self.name = name
        self.age = age 

p = Person("TianCheng".27) 
print("p:", p)
Copy the code

Output:

in __init__
p: <__main__.Person object at 0x105a689e8>
Copy the code

Understand that __init__ is responsible for the initialization work, usually also we often use.

1.2 __new__

Constructor: __new__(CLS, [… __new__ is the first function called in Python when an object is instantiated, before __init__. __new__ takes class as its first argument and returns an instance of the class. __init__ takes instance as an argument and initializes that instance. The __new__ function is called when each instance is created. Here’s an example:

class Person(object):
    def __new__(cls, *args, **kwargs):
        print("in __new__")
        instance = super().__new__(cls)
        return instance

    def __init__(self, name, age):
        print("in __init__")
        self._name = name
        self._age = age

p = Person("TianCheng".27)
print("p:", p)
Copy the code

Output result:

in __new__
in __init__
p: <__main__.Person object at 0x106ed9c18>
Copy the code

As you can see, the new method is used to create the object, and then init to initialize it. Suppose the __new__ method does not return the object, what happens?

class Person(object):
    def __new__(cls, *args, **kwargs):
        print("in __new__")
        instance = super().__new__(cls)
        #return instance

    def __init__(self, name, age):
        print("in __init__")
        self._name = name
        self._age = age

p = Person("TianCheng".27)
print("p:", p)

# output:
in __new__
p: None
Copy the code

It turns out that init can’t be initialized if new doesn’t return an instantiated object.

How to use the new method to implement singletons

class SingleTon(object):
    def __new__(cls, *args, **kwargs):
        if not hasattr(cls, "_instance"):
            cls._instance = cls.__new__(cls, *args, **kwargs)
        return cls._instance

s1 = SingleTon()
s2 = SingleTon()
print(s1)
print(s2)
Copy the code

Output result:

<__main__.SingleTon object at 0x1031cfcf8>
<__main__.SingleTon object at 0x1031cfcf8>
Copy the code

S1 and S2 have the same memory address to achieve singleton effect.

1.3 __call__

The __call__ method, first need to understand what is callable object, usually custom functions, built-in functions and classes are callable objects, but can be applied to a pair of parentheses () on an object can be called callable object, determine whether the object is callable object can use the function callable. Examples are as follows:

class A(object):
    def __init__(self):
        print("__init__ ")
        super(A, self).__init__()

    def __new__(cls):
        print("__new__ ")
        return super(A, cls).__new__(cls)

    def __call__(self):  You can define any parameter
        print('__call__ ')

a = A()
a()
print(callable(a))  # True
Copy the code

Output:

__new__
__init__
__call__
True
Copy the code

A () prints __call__. A is both an instantiated object and a callable object.

1.4 __del__

The __del__ destructor, which is executed when an object is deleted and automatically called when the object is destroyed in memory. For example:

class People:
    def __init__(self,name,age):
        self.name=name
        self.age=age

    def __del__(self): Execute automatically if the object is deleted
        print('__del__')

obj=People("Tiancheng".27)
#obj #obj.__del__() # execute __del__ directly in case of first deletion
Copy the code

Output result:

__del__
Copy the code

2. Closures and introspection

2.1 the closure

2.1.1 What closure

Simply put, an inner function is considered a closure if it refers to a variable in an outer (but not global) scope. Here’s a simple example:

>>>def addx(x):
>>>    def adder(y): return x + y
>>>    return adder
>>> c =  addx(8)
>>> type(c)
<type 'function'>
>>> c.__name__
'adder'
>>> c(10)
18
Copy the code

Where the adder(y) function is the closure.

2.1.2 Implement a closure and modify external variables

def foo(a):
    a = 1
    def bar(a):
        a = a + 1
        return a
    return bar
c = foo()
print(c())
Copy the code

I have a little example above, the purpose is to execute once, a increments by one, is it correct? The following error is reported.

local variable 'a' referenced before assignment
Copy the code

The reason is that the bar() function takes a as a local variable, and bar does not declare a. If the interviewer asks you how to change the value of a in Python2 and Python3. Python3 simply introduces the nonlocal keyword:

def foo(a):
    a = 1
    def bar(a):
        nonlocal a
        a = a + 1
        return a
    return bar
c = foo()
print(c()) # 2
Copy the code

Python2 does not have the nonlocal keyword.

def foo(a):
    a = [1]
    def bar(a):
        a[0] = a[0] + 1
        return a[0]
    return bar
c = foo()
print(c()) # 2
Copy the code

This is implemented with mutable variables, such as dict and list objects.

A common scenario for closures is decorators. We’ll talk about that later.

2.2 Introspection (Reflection)

Introspection, or reflexes, is often used in computer programming to refer to the ability to examine something to determine what it is, what it knows and what it can do. The main methods related to it:

  • Hasattr (Object, Name) Checks whether an object hasa specific name attribute. Returns a bool.
  • Getattr (Object, Name, default) Gets the name attribute of the object.
  • Setattr (Object, Name, default) sets the name attribute for an object
  • Delattr (object, name) Deletes the name attribute of an object
  • Dir ([object]) gets most of the attributes of the object
  • Isinstance (name, object) Checks whether name is an object
  • Type (object) Displays the type of an object
  • Callable (object) Determines whether an object is callable
>>> class A:
...   a = 1
...
>>> hasattr(A, 'a')
True
>>> getattr(A, 'a')
1
>>> setattr(A, 'b', 1)
>>> getattr(A, 'b')
1
>>> delattr(A, 'b')
>>> hasattr(A, 'b')
False
>>> dir(A)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'a']
>>> isinstance(1, int)
True
>>> type(A)
<class 'type'>
>>> type(1)
<class 'int'>
>>> callable(A)
True
Copy the code

Decorators and iterators

3.1 a decorator

A decorator is essentially a Python function or class that allows other functions or classes to add additional functionality without making any code changes (the decorator pattern in design mode), and the return value of the decorator is also a function/class object. It is often used in scenarios with faceted requirements, such as logging insertion, performance testing, transaction processing, caching, and permission verification.

3.1.1 Simple decorators

Let’s look at an example of a previous closure:

def my_logging(func):

    def wrapper(a):
        print("{} is running.".format(func.__name__))
        return func()  # func() is the same as foo() when foo is passed in as an argument
    return wrapper

def foo(a):
    print("this is foo function.")

foo = my_logging(foo)  Because the decorator my_logging(foo) returns a function object wrapper, this statement is equivalent to foo = wrapper
foo() Executing foo is equivalent to executing wrapper
Copy the code

But with the @ syntax sugar in Python, you can do this directly:

def my_logging(func):

    def wrapper(a):
        print("{} is running.".format(func.__name__))
        return func()
    return wrapper

@my_logging
def foo(a):
    print("this is foo function.")

foo()
Copy the code

Both of the above results will be printed as follows:

foo is running.
this is foo function.
Copy the code

My_logging is a decorator. It’s a normal function that wraps the func function that performs the real business logic in it. It looks like foo is decorated with my_logging and my_logging returns a function called Wrapper. In this case, the functions that enter and exit are called a cross section, and this approach to programming is called section-oriented programming (AOP).

If foo has parameters, how do you get them into the wrapper?

def my_logging(func):

    def wrapper(*args, **kwargs):
        print("{} is running.".format(func.__name__))
        return func(*args, **kwargs)
    return wrapper

@my_logging
def foo(x, y):
    print("this is foo function.")
    return x + y

print(foo(1.2))
Copy the code

**kwargs, **kwargs, ** args, **kwargs

foo is running.
this is foo function.
3
Copy the code

3.1.2 Decorators with parameters

The syntax of the decorator allows us to provide additional arguments, such as @decorator(a), when called. This greatly increases flexibility. For example, in log alarm scenarios, you can set alarm levels based on different alarms, such as INFO and WARN.

def my_logging(level):
    def decorator(func):
        def wrapper(*args, **kwargs):
            if level == "info":
                print("{} is running. level: ".format(func.__name__), level)
            elif level == "warn":
                print("{} is running. level: ".format(func.__name__), level)
            return func(*args, **kwargs)
        return wrapper
    return decorator

@my_logging(level="info")
def foo(name="foo"):
    print("{} is running".format(name))

@my_logging(level="warn")
def bar(name="bar"):
    print("{} is running".format(name))

foo()
bar()
Copy the code

Result output:

foo is running. level:  info
foo is running
bar is running. level:  warn
bar is running
Copy the code

My_logging above is a decorator that allows arguments. It is actually a function wrapper around the original decorator and returns a decorator. We can think of it as a closure with parameters. When called with @my_logging(level=”info”), Python can detect this layer of encapsulation and pass arguments to the decorator’s environment. @my_logging(level=”info”) is equivalent to @decorator

3.1.3 Class decorators

Decorators can not only be functions, but also classes. Compared with function decorators, class decorators have the advantages of greater flexibility, higher cohesion and encapsulation. Using class decorators relies heavily on the class’s __call__ method, which is called when a decorator is attached to a function using the @ form.

class MyLogging(object):

    def __init__(self, func):
        self._func = func

    def __call__(self, *args, **kwargs):
        print("class decorator starting.")
        a = self._func(*args, **kwargs)
        print("class decorator end.")
        return a

@MyLogging
def foo(x, y):
    print("foo is running")
    return x + y

print(foo(1.2))
Copy the code

Output result:

class decorator starting.
foo is running
class decorator end.
3
Copy the code

3.1.4 functools. Wraps

In Python, there is also a decorator, functools.wraps. First, there is a problem, because the original function is decorated by the decorator function, something happens:

def my_logging(func):

    def wrapper(*args, **kwargs):
        print("{} is running.".format(func.__name__))
        return func(*args, **kwargs)
    return wrapper

@my_logging
def foo(x, y):
    """ add function """
    print("this is foo function.")
    return x + y

print(foo(1.2))
print("func name:", foo.__name__)
print("doc:", foo.__doc__)
Copy the code

Print result:

foo is running.
this is foo function.
3
func name: wrapper
doc: None
Copy the code

The func name should print foo, and doc is not None. It is found that after the original function is decorated by the ornament function, the meta information changes, which is obviously not what we want. In Python, we can use functools.wraps to solve this problem, so as to preserve the original function meta information.

from functools import wraps

def my_logging(func):

    @wraps(func)
    def wrapper(*args, **kwargs):
        print("{} is running.".format(func.__name__))
        return func(*args, **kwargs)
    return wrapper

@my_logging
def foo(x, y):
    """ add function """
    print("this is foo function.")
    return x + y

print(foo(1.2))
print("func name:", foo.__name__)
print("doc:", foo.__doc__)
Copy the code

Output result:

foo is running.
this is foo function.
3
func name: foo
doc:
    add function
Copy the code

3.1.5 Execution sequence of multiple decorators

@a
@b
@c
def f (a):
    pass
Copy the code

F = a(b(c(f))

3.2 Iterators VS generators

Let’s start with a diagram:

3.2.1 Container

A container is a data structure that organizes multiple elements together. Elements in a Container can be obtained iteratively one by one. You can use the in and not in keywords to determine whether elements are contained in the container. Common Container objects in Python are:

list, deque, ....
set, frozensets, .... dict, defaultdict, OrderedDict, Counter, .... The tuple, namedtuple,... strCopy the code

For example:

>>> assert 1 in [1, 2, 3]      # lists
>>> assert 4 not in [1, 2, 3]
>>> assert 1 in {1, 2, 3}      # sets
>>> assert 4 not in {1, 2, 3}
>>> assert 1 in (1, 2, 3)      # tuples
>>> assert 4 not in (1, 2, 3)
Copy the code

3.2.2 Iterables vs Iterators

Most Containers are iterable. For example, a list or set is an iterable. Any container that returns an iterator is an iterable. Here’s an example:

>>> x = [1, 2, 3]
>>> y = iter(x)
>>> next(y)
1
>>> next(y)
2
>>> type(x)
<class 'list'>
>>> type(y)
<class 'list_iterator'>
Copy the code

So x is an iterable, which is also called a container. Y is an iterator and implements the __iter__ and __next__ methods. The relationship between them is:

3.2.3 Generators

A generator must be an iterator, a special kind of iterator, special in that it doesn’t need the __iter__() and next methods above, just a yiled keyword. Here’s an example: Implementing Fibonacci with a generator:

# content of test.py
def fib(n):
    prev, curr = 0.1
    while n > 0:
        yield curr
        prev, curr = curr, curr + prev
        n -= 1
Copy the code

Run the FIB function on the terminal:

>>> from test import fib
>>> y = fib(10)
>>> next(y)
1
>>> type(y)
<class 'generator'>
>>> next(y)
1
>>> next(y)
2
Copy the code

Fib is just a normal Python function. It is special in that there is no return keyword in the function body, and the return value of the function is a generator object (via the yield keyword). When f=fib() returns a generator object, the code in the function body is not executed, but only when next is explicitly or implicitly called. Suppose there are tens of thousands of objects that need to be fetched sequentially. If they are loaded into the memory at one time, it will be a great pressure on the memory. After having a generator, one can be generated when needed, and the one that is not needed will not occupy the memory.

You might also encounter generator expressions such as:

>>> a = (x*x for x in range(10))
>>> a
<generator object <genexpr> at 0x102d79a20>
>>> next(a)
0
>>> next(a)
1
>>> a.close()
>>> next(a)
Traceback (most recent call last):
  File "<stdin>", line 1.in <module>
StopIteration
Copy the code

These tips are also very useful. Close turns off the generator. There is also a send method in the generator, where send(None) is equivalent to next.

>>> def double_inputs():
...     while True:
...         x = yield
...         yield x * 2
...
>>> generator = double_inputs()
>>> generator.send(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't send non-None value to a just-started generator
>>> generator.send(None)
>>> generator.send(10)
20
>>> next(generator)
>>> generator.send(20)
40
Copy the code

As you can see from the above example, the generator can receive arguments through the send(value) method, and the first time it cannot send(value) directly, it needs to send(None) or next() after execution. That is, the generator must be suspended until send is called and a value other than None is passed in, otherwise an exception will be thrown.

3.2.4 Differences between iterators and generators

Maybe after you read the above, you are wondering what is the difference between them?

  • An iterator is a more abstract concept. Any object that has a next method (next python3, python2 is the __next__ method) and an __iter__ method can be called an iterator.

  • Each generator is an iterator, but not the other way around. Typically, generators are generated by calling a function s consisting of one or more yield expressions. Satisfies the definition of an iterator.

  • Generators can do everything an iterator can do, and because the iter() and next() methods are created automatically, generators are particularly compact and efficient.

You can check out the Python interview details at gitbook.cn/gitchat/act…

More exciting articles please pay attention to the public account: “Tiancheng Technology Miscellaneous talk”