The following is a reading note for Effective Python: 90 Effective Ways to Write High-quality Python Code (original book, 2nd Edition).

Please note that all information mentioned in the book is not guaranteed

Develop Pythonic thinking

usef'{var}Implement formatted string output

  • using%Operators have a lot of problems filling values into C-style format strings, and it can be cumbersome to write.
  • str.formatMethod defines its format specifiers in a mini-language that gives us some useful concepts, but otherwise suffers from many of the same shortcomings as c-style format strings, so we should avoid it as well.
  • f-stringThe biggest problem with c-style format strings is solved by using a new writing method that fills values into strings.f-stringA compact and powerful mechanism for embedding arbitrary Python expressions directly into format specifiers.

See: weread.qq.com/web/reader/…

scores = {'Andy': 100.'Bob': 90.'Cici': 80}
name = 'Andy'

Use the % format operator
print('name=%s, score=%d' % (name, scores[name]))

# Use the format function: do not set the specified position, the default order
print("name={}, score={}".format(name, scores[name]))

# Use the format function: set the specified position
print("name={0}, score={1}".format(name, scores[name]))

# Use the format function: set parameters through the dictionary
print("name={name}, score={score}".format(* * {'name': name, 'score': scores[name]}))

# use interpolated format strings
print(f"name={name}, score={scores[name]}")


# name=Andy, score=100
# name=Andy, score=100
# name=Andy, score=100
# name=Andy, score=100
# name=Andy, score=100
Copy the code

Lists and dictionaries

Data resolution

  • unpackingIs a special Python syntax that allows multiple values in a data structure to be assigned to variables in a single line of code.
  • unpackingIn Python, any iterable object can be split up, no matter how many layers of iterative structure it contains.
  • As far as possible byunpackingTo unpack the data in the sequence instead of accessing it via subscripts, which makes the code cleaner and cleaner.

See: weread.qq.com/web/reader/…

  • When you break up a data structure and assign its data to variables, you can use asterisked expressions to capture the contents of the structure that do not match normal variables into a list.
  • This asterisked expression can appear anywhere to the left of the assignment symbol and will always form a list of zero or more values.
  • This asterisked method of unpacking is clear when unpacking a list into non-overlapping parts, while subscripting and slicing is error-prone.

See: weread.qq.com/web/reader/…

  • When returning multiple values, you can use an asterisk expression to receive values that are not captured by normal variables
  • A function can combine multiple values and return them to the caller in a tuple that can be unpacked using Python’s unpacking mechanism.
  • For multiple values returned by a function, you can capture all those values that are not captured by normal variables in a single variable marked with an asterisk.
  • It is very error-prone to split the returned value into four or more variables, so it is best not to do so and should be done through a small class or namedTuple instance.

See: weread.qq.com/web/reader/…

names = ('Andy'.'Bob'.'Cici'.'Daniel')
name1, name2, *others = names
print(f'names = {names}')
print(f'|- name1 = {name1}')
print(f'|- name2 = {name2}')
print(f'|- others = {others}')

# names = ('Andy', 'Bob', 'Cici', 'Daniel')
# |- name1 = Andy
# |- name2 = Bob
# |- others = ['Cici', 'Daniel']
Copy the code

In Python code, variables that are not needed can be represented by underscores

Data split – implement exchange

a = 111; b = 222
print(F 'before swapping: a={a}, b={b}')
a, b = b, a
print(After f' is swapped: a={a}, b={b}')

A =111, b=222
A =222, b=111
Copy the code

As far as possible withenumerate

  • The enumerate function iterates over the iterator in concise code and indicates the sequence number of the current cycle.
  • Instead of specifying the range of subscripts through range and then using the subscripts to access the sequence, you should simply iterate through the enumerate function.
  • You can specify the start sequence number (0 by default) using the second parameter to enumerate.

See: weread.qq.com/web/reader/…

List:

tuple_list = [('AAA'.'111'), ('BBB'.'222'), ('CCC'.'333')]
for index, (key, value) in enumerate(tuple_list):
    print(f'#{index}: {key} = {value}')

# #0: AAA = 111
# #1: BBB = 222
# #2: CCC = 333
Copy the code

Dictionary:

dict = {'AAA': '111'.'BBB': '222'.'CCC': '333'}

print('\nenumerate(dict):')
for index, key in enumerate(dict) :print(f'#{index}: {key} = {dict.get(key)}')

print('\nenumerate(dict.items()):')
for index, (key, value) in enumerate(dict.items()):
    print(f'#{index}: {key} = {value}')

# enumerate(dict):
# #0: AAA = 111
# #1: BBB = 222
# #2: CCC = 333
# 
# enumerate(dict.items()):
# #0: AAA = 111
# #1: BBB = 222
# #2: CCC = 333
Copy the code

withzipIterate over multiple collections at the same time

  • The built-in ZIP function iterates through multiple iterators at the same time.
  • Zip creates a lazy generator that generates only one tuple at a time, so it processes the input one at a time, no matter how long it is.
  • If the iterators provided are inconsistent in length, zip will stop as soon as any of them are iterated.
  • If you want to iterate through the longest iterator, use the zip_longest function in the built-in itertools module instead.

See: weread.qq.com/web/reader/…

names = ['Andy'.'Bob'.'Cici'.'Daniel']
scores = [100.90.80]

print(If the length of two sets is different, then the shorter set will complete the whole traversal:)
for name, score in zip(names, scores):
    print(f'{name}: {score}')

import itertools
print('n# or use itertools.zip_longest() to iterate over longer sets:)
for name, score in itertools.zip_longest(names, scores):
    print(f'{name}: {score}')

If the length of two sets is not the same, then when the shorter set is traversed, the whole traversal will be completed:
# Andy: 100
# Bob: 90
# Cici: 80
#
Itertools.zip_longest () :
# Andy: 100
# Bob: 90
# Cici: 80
# Daniel: None
Copy the code

Custom sort

  • The sort method of a list sorts its strings, integers, tuples, and other elements of built-in type in natural order.
  • Ordinary objects can also be sorted using the sort method if they have a natural order defined in a special way, but such objects are rare.
  • You can pass the helper function to the key argument of sort and have sort sort the elements by the value returned by the function rather than by the elements themselves.
  • If there are many items to sort by, you can put them in a tuple and have the key function return such a tuple.
  • For types that support the unary subtraction operator, you can invert this index separately and have the sorting algorithm work in the opposite direction on this index.
  • If these metrics do not support the unary subtraction operator, you can call the sort method multiple times and specify the key function and reverse parameters separately on each call.
  • The least important indicators are dealt with in the first round, then the more important indicators are dealt with gradually, and the primary indicators are dealt with in the last round.

See: weread.qq.com/web/reader/…

class Student:
    name = ' '
    score = 0


    def __init__(self, name, score) :
        self.name = name
        self.score = score


    def __repr__(self) :
        return f'Student({self.name}.{self.score}) '


print(F 'before sorting:')
students = [Student('Bob'.60), Student('Andy'.100), Student('Cici'.80), Student('Bob'.80)]
for stu in students: print(f'|- {stu.name}: {stu.score}')

print(F '\n after sorting by name ascending: ')
students.sort(key=lambda stu: stu.name)
for stu in students: print(f'|- {stu.name}: {stu.score}')

print(F '\n After sorting by name: ')
students.sort(key=lambda stu: stu.name, reverse=True)
for stu in students: print(f'|- {stu.name}: {stu.score}')

print(F '\n is sorted in reverse order by name, and those with the same name have the highest score first: ')
students.sort(key=lambda stu: (stu.name, stu.score), reverse=True)
for stu in students: print(f'|- {stu.name}: {stu.score}')

print(F '\n after descending order by fraction: ')
students.sort(key=lambda stu: stu.score, reverse=True)
for stu in students: print(f'|- {stu.name}: {stu.score}')

# before sorting:
# |- Bob: 60
# |- Andy: 100
# |- Cici: 80
# |- Bob: 80
# 
# after sorting by name:
# |- Andy: 100
# |- Bob: 60
# |- Bob: 80
# |- Cici: 80
# 
After sorting by name:
# |- Cici: 80
# |- Bob: 60
# |- Bob: 80
# |- Andy: 100
# 
# order in reverse order by name and score first with the same name:
# |- Cici: 80
# |- Bob: 80
# |- Bob: 60
# |- Andy: 100
# 
# after sorting in descending order:
# |- Andy: 100
# |- Cici: 80
# |- Bob: 80
# |- Bob: 60
Copy the code

Use the dictionary to get the attribute key-value pairs of the class

  • In Python 3.5 and earlier, the order you see when iterating through dict seems arbitrary

  • Since Python 3.6, dictionaries have retained the order in which these key-value pairs are added, and the Python 3.7 language specification formalized this rule.

    Thus, in a new version of Python, it is always possible to iterate over these key-value pairs in the same order as when the dictionary was created.

  • Classes also use dictionaries to hold some data held by instances of the class, and in the new version of Python we can expect these fields to appear in __dict__ in the same order as they were assigned.

  • The Current Python language specification already requires that dictionaries retain the order in which key-value pairs are added. So, we can use this feature to implement some functionality, and we can incorporate it into the apis we design for our classes and functions.

class Student:
    name = ' '
    score = 0
    id = '001'
    _sex = None
    __id_num = None

    def __init__(self, name, score) :
        self._sex = 'M'
        self.name = name
        self.score = score

    def __repr__(self) :
        return f'Student({self.name}.{self.score}) '


students = [Student('Bob'.60), Student('Andy'.100), Student('Cici'.80), Student('Bob'.80)]
print('# through students')
for stu in students: print(f'|- {stu.name}: {stu.score}')

print('\n# traverses the students property key pair ')
for stu in students: print(f'|- {stu.__dict__}')

# # pass through students
# |- Bob: 60
# |- Andy: 100
# |- Cici: 80
# |- Bob: 80
# 
# # Iterate over the students property key value pair
# |- {'_sex': 'M', 'name': 'Bob', 'score': 60}
# |- {'_sex': 'M', 'name': 'Andy', 'score': 100}
# |- {'_sex': 'M', 'name': 'Cici', 'score': 80}
# |- {'_sex': 'M', 'name': 'Bob', 'score': 80}
Copy the code

Unordered dictionary & ordered dictionary

  • Starting with Python version 3.7, we can be sure that the order we see when iterating over a standard dictionary is the same as the order in which these key-value pairs were inserted.

  • In Python code, it’s easy to define objects that look like standard dictionaries but aren’t dict instances themselves.

  • For objects of this type, it cannot be assumed that the order seen during iteration is necessarily the same as the order seen during insertion.

  • If you don’t want to treat a type that is very similar to a standard dictionary as a standard dictionary, there are three ways to do this.

  • First, don’t rely on the order in which you inserted your code; Second, determine whether the program is a standard dictionary at run time. Third, add type annotations to the code and do static analysis.

  • In fact, the built-in Collections module has long provided such a dictionary with insertion order, called OrderedDict.

  • It behaves much like standard dict types (since Python 3.7), but has a big difference in performance.

  • OrderedDict may be more appropriate than the standard Pythondict type if key-value pairs are frequently inserted or ejected (for example, to implement least-recently-used caching)

See: weread.qq.com/web/reader/…

import collections
import os

print(F '# python env: ', end=' ')
os.system("python3 --version")

scores = dict()
scores['Bob'] = 60
scores['Andy'] = 100
scores['Cici'] = 80
print('\ n# traversal scores')
for name, score in scores.items(): print(f'|- {name}: {score}')

ordered_scores = collections.OrderedDict()
ordered_scores['Bob'] = 60
ordered_scores['Andy'] = 100
ordered_scores['Cici'] = 80
print('\ n# traversal ordered_scores')
for name, score in ordered_scores.items(): print(f'|- {name}: {score}')

Python env: Python 3.8.9
# 
Iterate over scores
# |- Bob: 60
# |- Andy: 100
# |- Cici: 80
# 
Iterate over ordered_scores
# |- Bob: 60
# |- Andy: 100
# |- Cici: 80
Copy the code

Dictionary values, assignments, and defaults

See: weread.qq.com/web/reader/…

In addition defaultdict visible: weread.qq.com/web/reader/…

scores = {'Andy': 100.'Bob': 90.'Cici': 80}
name = 'Daniel'
score = None

# Traditional methods
if name in scores:
    score = scores[name]
print(f"{name}'s score is {score}")

# Recommended methods
score2 = scores.get(name, 0)
print(f"{name}'s score is {score2}")

# set the default value:
This method will query the dictionary for the key and return the corresponding value if there is one; If not, associate the user-supplied default value with the key and insert it into the dictionary, then return the value.
scores.setdefault(name, 60)
print(f"{name}'s score is {scores.get(name)}")

# Daniel's score is None
# Daniel's score is 0
# Daniel's score is 60
Copy the code

Dictionary value – User-defined__missing__

  • If default values to construct must be determined by key names, you can define your own dict subclass and implement it__missing__Methods.

See: weread.qq.com/web/reader/…

class Scores(dict) :

    def __missing__(self, key) :
        self[key] = None


# Traditional methods
scores = dict(a)try:
    print(f"xxx's score is {scores['xxx']}")  # KeyError: 'xxx'
except KeyError as err:
    print(f'KeyError: {err}')

# Solve this problem by inheriting dict types and implementing the __missing__ special method
scores2 = Scores()
print(f"xxx's score is {scores2['xxx']}")

# KeyError: 'xxx'
# xxx's score is None
Copy the code

A dictionary that supports custom __missing__ :

class safedict(dict) :
    missing = None

    def __missing__(self, key) :
        if self.missing:
            return self.missing(key)
        return None


empty_dict = safedict()
empty_dict.missing = lambda key: f'key:{key} not define.'
print(f'{empty_dict["qqq"]}')

# key:qqq not define.
Copy the code

function

Use variables outside of scope

  • Closures can refer to variables in the enclosing scope in which they are defined.
  • As written by default, assigning a value to a variable inside a closure does not change the variable of the same name in the enclosing scope.
  • Variables in peripheral scopes can be modified by specifying them using nonlocal statements and then assigning values.
  • Use nonlocal statements sparingly, except for very simple functions.

See: weread.qq.com/web/reader/…

Let’s start with an example:

def fun_outer() :
    outer_var = None

    def func_inner() :
        outer_var = 'new value'
        return outer_var

    func_inner()
    return outer_var


print(f'value = {fun_outer()}')
# value = None
Copy the code

In the code above, value = None instead of value = new value

We can do this by using nonlocal, which “assigns data inside the closure to variables outside the closure” :

def fun_outer() :
    outer_var = None

    def func_inner() :
        nonlocal outer_var  # key
        outer_var = 'new value'
        return outer_var

    func_inner()
    return outer_var


print(f'value = {fun_outer()}')
# value = new value
Copy the code

Variable parameter

  • withdefWhen you define a function, you can pass*argsIs written so that the function accepts a variable number of positional arguments.
  • When you call a function, you can add to the left of the sequence*Operator that passes the element as a positional argument*argsThis part right over here.
  • if*The operator is appended to the generator so that when passing parameters, the program may crash because it runs out of memory.
  • To accept*argsAdd new positional parameters to the function of, which can lead to hard-to-troubleshoot bugs.

See: weread.qq.com/web/reader/…

def func(label, *args) :
    if not args:
        print(f'{label}')
    else:
        print(f'{label}: {",".join(args)}')


func('hello')
func('hello'.'Andy'.'Bob'.'Cici'.'Daniel')

# hello
# hello: Andy, Bob, Cici, Daniel
Copy the code

Keyword parameter

  • Function arguments can be specified by position or in the form of keywords.
  • The keyword makes it clear what each argument does, because specifying arguments by position when calling a function can lead to ambiguous arguments.
  • You should extend the behavior of a function with keyword arguments with default values, because this does not affect the original function calling code.
  • Optional keyword arguments should always be passed by parameter name, not position.

See: weread.qq.com/web/reader/…

def get_url(host='http://baidu.com', path=None, **kwargs) :
    return f"{host}/{path or ' '}{'? ' if kwargs else ' '}{'&'.join([f'{key}={value}' for key, value in kwargs.items()])}"


print(get_url())
print(get_url(path='aaa/bbb'))
print(get_url(path='aaa/bbb', k1='v1', k2='v2'))
print(get_url('http://google.com'))
print(get_url('http://google.com', path='aaa/bbb'))
print(get_url('http://google.com', path='aaa/bbb', k1='v1', k2='v2'))

# http://baidu.com/
# http://baidu.com/aaa/bbb
# http://baidu.com/aaa/bbb?k1=v1&k2=v2
# http://google.com/
# http://google.com/aaa/bbb
# http://google.com/aaa/bbb?k1=v1&k2=v2
Copy the code

A mixture of positional, variable, and keyword arguments

def func(a, b, c='c_default', *args, **kwargs) :
    print(f'a={a}, b={b}, c={c}, ', end=' ')
    for i, v in enumerate(args): print(f'args[{i}] ={v}, ', end=' ')
    for k, v in kwargs.items(): print(f'{k}={v}, ', end=' ')
    print()


func('1'.'2')
func('1'.'2'.'3')
func('1'.'2', c='3')
func('1'.'2'.'3'.'4'.'5'.'6')
func('1'.'2'.'3'.'4'.'5'.'6', x='7', y='8', z='9')
func('1'.'2', x='7', y='8', z='9')

# a=1, b=2, c=c_default, 
# a=1, b=2, c=3, 
# a=1, b=2, c=3, 
# a=1, b=2, c=3, args[0]=4, args[1]=5, args[2]=6, 
# a=1, b=2, c=3, args[0]=4, args[1]=5, args[2]=6, x=7, y=8, z=9, 
# a=1, b=2, c=c_default, x=7, y=8, z=9, 
Copy the code

Set the default values of the parameters appropriately

  • The default value of an argument is computed only once, when the system loads the module that defines the function.
  • So, if the default values may be changed by the caller in the future (e.g. {}, []) or vary with the situation at the time of the call (e.g. Datetime.now ()), the program can have strange effects.
  • If the default value of the keyword argument is one of these variable values, it should be None, and the default behavior of the function should be described in the docString.
  • Keyword arguments that default to None. Type annotations can also be added.

See: weread.qq.com/web/reader/…

from datetime import datetime
import time
from typing import Optional


# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = error case = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
def log_1(message, time=datetime.now()) :
    print(f'[{time}] {message}')


log_1('AAAAA')
time.sleep(0.5)
log_1('BBBBB')


# The time of the two logs is the same, which is not expected
[the 2022-04-14 11:28:17. # 523949] AAAAA
[the 2022-04-14 11:28:17. # 523949] BBBBB

# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = to fix the problem = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
def log_2(message, time=None) :
    print(f'[{time or datetime.now()}] {message}')


log_2('CCCCC')
time.sleep(0.5)
log_2('DDDDD')


# As expected
[the 2022-04-14 11:30:25. # 636013] CCCCC
[the 2022-04-14 11:30:26. # 139351] DDDDD

# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = optimization = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
# optimizer: Specify the parameter type
def log_3(message: str, time: Optional[datetime] = None) :
    print(f'[{time or datetime.now()}] {message}')


log_3('EEEEE')
time.sleep(0.5)
log_3('FFFFF')

[the 2022-04-14 11:34:15. # 135794] EEEEE
# FFFFF [the 2022-04-14 11:34:15. 638360]
Copy the code

Use positional and keyword arguments wisely

  • Keyword-only argumentIs an argument that can only be specified by keyword and not by location.
    • This forces the caller to specify which parameter the value is passed to.
    • In the argument list of a function, the argument is located*To the right of the symbol.
  • Positional-only argumentIs an argument that does not allow the caller to specify it by keyword, but requires that it be passed by location.
    • This reduces the coupling between the calling code and the parameter name.
    • In the argument list of a function, these arguments are to the left of the/symbol.
  • In the parameter list, is located/with*Can be specified by position or by keyword.
    • This is also the default way to specify normal Python arguments.

See: weread.qq.com/web/reader/…

# can be passed as either a positional or a keyword parameter
This makes it possible to change the parameter name to affect the caller
def func_1(a, b, c, d) :
    print(f'a={a}, b={b}, c={c}, d={d}')


func_1(1.2.3.4)
func_1(1.2, c=3, d=4)
func_1(a=1, b=2, c=3, d=4)


# a=1, b=2, c=3, d=4
# a=1, b=2, c=3, d=4
# a=1, b=2, c=3, d=4

# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

# insert '/' and '*' to specify that "arguments can only be passed in a certain way", i.e. in the argument list of the function:
The arguments to the left of the # / symbol are positional only arguments, and the arguments to the right of the * symbol are keyword only arguments.
The argument between the two symbols can be provided by position or specified as a keyword
def func_2(a, b, /, c, *, d) :
    print(f'a={a}, b={b}, c={c}, d={d}')


func_2(1.2.3, d=4)
func_2(1.2, c=3, d=4)

# a=1, b=2, c=3, d=4
# a=1, b=2, c=3, d=4
Copy the code

Function modifier

  • A decorator is a Python method of wrapping a function inside another function so that the program has a chance to execute some logic before and after executing the original function.
  • Modifiers can cause strange behavior in tools that exploit the introspection mechanism, such as debuggers.
  • The built-in Functools module in Python has a decorator called Wraps that will help you define your decorators correctly to avoid the problem.

See: weread.qq.com/web/reader/…

Function modifiers are not used
def log_1(*args) :
    print(f'{",".join(args)}')


log_1('hello')
log_1('hello'.'world')
log_1('hello'.'world'.'~ ~')

# hello
# hello, world
# hello, world, ~~

# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

from functools import wraps
from datetime import datetime


Define the function modifier
def with_time(func) :
    @wraps(func)
    def wrapper(*args, **kwargs) :
        print(f'[{datetime.now()}] ', end=' ')
        result = func(*args, **kwargs)
        return result
    return wrapper


Use the @ symbol to apply the modifier to the function you want to debug
@with_time
def log_2(*args) :
    print(f'{",".join(args)}')


print('\n')
log_2('hello')
log_2('hello'.'world')
log_2('hello'.'world'.'~ ~')

[the 2022-04-14 17:35:16. # 344146] hello
# [2022-04-14 17:35:16.344173] # [2022-04-14 17:35:16.344173
# [2022-04-14 17:35:16.344186] hello, world, ~~
Copy the code