5 years of Python skills, summarized 10 development tips! Net friend: Too practical

Hello, everyone

Today I’m going to share 10 Python development tips that I’ve compiled that are very useful.

It’s worth noting that all 10 of these tips are included in my own Python Guide to Dark Magic

You can send “Dark Magic” in the background to get a beautifully formatted PDF ebook.

1. How do I view source code in the running state?

Look at the source code of the function, which we usually do using an IDE.

In PyCharm, for example, you can Ctrl + mouse click to access the source code of the function.

What if there is no IDE?

When we want to use a function, how do we know what parameters the function needs to take?

When something goes wrong with a function, how do we read the source code to troubleshoot the problem?

In this case, we can use Inspect instead of the IDE to help you do this

# demo.py
import inspect


def add(x, y):
    return x + y

print("= = = = = = = = = = = = = = = = = = =")
print(inspect.getsource(add))
Copy the code

The result is as follows

$ python demo.py
===================
def add(x, y):
    return x + y
Copy the code

2. How do I disable the automatic association context?

When you throw another exception while handling an exception, due to mishandling or other problems, the exception thrown will also carry the original exception information.

It looks something like this.

try:
    print(1 / 0)
except Exception as exc:
    raise RuntimeError("Something bad happened")
Copy the code

You can see two exception messages in the output

Traceback (most recent call last):
  File "demo.py", line 2.in <module>
    print(1 / 0)
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo.py", line 4.in <module>
    raise RuntimeError("Something bad happened")
RuntimeError: Something bad happened
Copy the code

If an exception is thrown in an exception handler or finally block, by default, the exception mechanism implicitly attaches the previous exception to the __context__ attribute of the new exception. This is the automatic correlation exception context that Python turns on by default.

If you want to control the context yourself, add the from keyword. (The from syntax has a limitation that the second expression must be another exception class or instance.) To indicate which exception is directly caused by your new exception.

try:
    print(1 / 0)
except Exception as exc:
    raise RuntimeError("Something bad happened") from exc
Copy the code

The output is as follows

Traceback (most recent call last):
  File "demo.py", line 2.in <module>
    print(1 / 0)
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "demo.py", line 4.in <module>
    raise RuntimeError("Something bad happened") from exc
RuntimeError: Something bad happened
Copy the code

Of course, you can also set the __context__ attribute for the exception with the with_traceback() method, which also makes it easier to display the exception information in traceback.

try:
    print(1 / 0)
except Exception as exc:
    raise RuntimeError("bad thing").with_traceback(exc)
Copy the code

Finally, what if I wanted to completely turn off the automatic correlation of exception context? What can be done?

You can use raise… From None, from the example below, there is no original exception

$ cat demo.py
try:
    print(1 / 0)
except Exception as exc:
    raise RuntimeError("Something bad happened") from None
$
$ python demo.py
Traceback (most recent call last):
  File "demo.py", line 4.in <module>
    raise RuntimeError("Something bad happened") from None
RuntimeError: Something bad happened
(PythonCodingTime)
Copy the code

03. Fastest way to view the package search path

When you use import to import a package or module, Python will look it up in some of the prioritized directories that a normal person would look at using sys.path.

>>> import sys
>>> from pprint import pprint   
>>> pprint(sys.path)
[' '.'/ usr/local/Python3.7 / lib/python37 zip'.'/ usr/local/Python3.7 / lib/Python3.7'.'/ usr/local/Python3.7 / lib/Python3.7 / lib - dynload'.'/ home/wangbm/local/lib/python3.7 / site - packages'.'/ usr/local/Python3.7 / lib/Python3.7 / site - packages']
>>> 
Copy the code

Is there a faster way?

Is there a way THAT I don’t even have to go into console mode?

You might expect this, but it’s essentially the same

[wangbm@localhost ~]$ python -c "print('\n'.join(__import__('sys').path))"

/usr/lib/python27./site-packages/pip18.1-py27..egg
/usr/lib/python27./site-packages/redis3.01.-py27..egg
/usr/lib64/python27.zip
/usr/lib64/python27.
/usr/lib64/python27./plat-linux2
/usr/lib64/python27./lib-tk
/usr/lib64/python27./lib-old
/usr/lib64/python27./lib-dynload
/home/wangbm/.local/lib/python27./site-packages
/usr/lib64/python27./site-packages
/usr/lib64/python27./site-packages/gtk2.0
/usr/lib/python27./site-packages
Copy the code

What I’m going to show you here is a much more convenient way to do it than either of the above methods, which can be solved with a single command

[wangbm @ localhost ~] $python3 - m site sys path = ['/home/wangbm ', '/ usr/local/Python3.7 / lib/python37 zip', '/ usr/local/Python3.7 / lib/Python3.7', '/ usr/local/Python3.7 / lib/Python3.7 / lib - dynload', '/ home/wangbm/local/lib/python3.7 / site - packages','/usr/local/python3.7 / lib/python3.7 / site - packages',] USER_BASE: '/ home/wangbm /. Local' (exists) USER_SITE: '/ home/wangbm/local/lib/python3.7 / site - packages' (exists) ENABLE_USER_SITE: TrueCopy the code

As you can see from the output, this column has a more complete path than sys.path and contains the directory for the user environment.

4. Write the nested for loop as a single line

We often have nested for loop code like this

list1 = range(1.3)
list2 = range(4.6)
list3 = range(7.9)
for item1 in list1:
    for item2 in list2:
      	for item3 in list3:
        	  print(item1+item2+item3)
Copy the code

There are only three for loops here, and in actual coding, there could be more layers.

This code is very unreadable, and many people don’t want to write it this way, but they don’t have a better way to write it.

Here’s an approach I use a lot, using the iterTools library for more elegant and readable code.

from itertools import product
list1 = range(1.3)
list2 = range(4.6)
list3 = range(7.9)
for item1,item2,item3 in product(list1, list2, list3):
    print(item1+item2+item3)
Copy the code

The output is as follows

$ python demo.py
12
13
13
14
13
14
14
15
Copy the code

5. How to use print to output logs

Beginners like to use print to debug code and record the program as it runs.

However, print only outputs the content to the terminal and cannot be persisted in the log file, which is not conducive to the troubleshooting of problems.

If you’re keen on using print to debug your code (although it’s not the best practice) and log your program as it runs, the following use of print might be useful.

Print in Python 3 is much more powerful as a function that accepts more arguments, specifying arguments to print the contents of print to a log file

The code is as follows:

>>> with open('test.log', mode='w') as f:
.    print('hello, python', file=f, flush=True)
>>> exit()

$ cat test.log
hello, python
Copy the code

6. How to calculate the running time of function quickly

To calculate the running time of a function, you might do something like this

import time

start = time.time()

# run the function

end = time.time()
print(end-start)
Copy the code

Look how many lines of code you’ve written to calculate the running time of the function.

Is there a way to calculate this running time more easily?

There is.

There is a built-in module called Timeit

To use it, you only need one line of code

import time
import timeit

def run_sleep(second):
    print(second)
    time.sleep(second)

# Use this line only
print(timeit.timeit(lambda :run_sleep(2), number=5))
Copy the code

The result is as follows

2
2
2
2
2
10.020059824
Copy the code

7. Use the built-in caching mechanism to improve efficiency

Cache is a processing method that saves quantitative data for meeting subsequent acquisition requirements, aiming to speed up data acquisition.

The data generation process may need to be calculated, structured, and remotely acquired. If the same data needs to be used several times, it will waste a lot of time to regenerate it every time. Therefore, caching data from operations such as calculations or remote requests can speed up subsequent data retrieval requirements.

To do this, Python 3.2 + provides a mechanism that can be easily implemented without requiring you to write such logical code.

This mechanism is implemented in the FUNCTool module’s Lru_cache decorator.

@functools.lru_cache(maxsize=None, typed=False)
Copy the code

Parameter interpretation:

Maxsize: The maximum number of calls to this function that can be cached. If it is None, it is unlimited. A power of 2 is best
Typed: If it is True, calls with different parameter types will be cached separately.

For example

from functools import lru_cache

@lru_cache(None)
def add(x, y):
    print("calculating: %s + %s" % (x, y))
    return x + y

print(add(1.2))
print(add(1.2))
print(add(2.3))
Copy the code

As you can see in the output below, the second call does not actually execute the function body, but instead returns the result directly in the cache

calculating: 1 + 2
3
3
calculating: 2 + 3
5
Copy the code

Here’s the classic Fibonacci sequence, where when you specify a large n, there’s a lot of double counting

def fib(n):
    if n < 2:
        return n
    return fib(n - 2) + fib(n - 1)
Copy the code

Timeit, introduced in point 6, can now be used to test how much efficiency can be improved.

If no lRU_cache is used, the runtime is 31 seconds

import timeit

def fib(n):
    if n < 2:
        return n
    return fib(n - 2) + fib(n - 1)



print(timeit.timeit(lambda :fib(40), number=1))
# the output: 31.2725698948
Copy the code

With lru_cache, it was running too fast, so I changed the n value from 30 to 500, and even then it only took 0.0004 seconds. The speed increase is very significant.

import timeit
from functools import lru_cache

@lru_cache(None)
def fib(n):
    if n < 2:
        return n
    return fib(n - 2) + fib(n - 1)

print(timeit.timeit(lambda :fib(500), number=1))
# the output: 0.0004921059880871326
Copy the code

8. The technique of executing code before program exit

Atexit is a built-in module that makes it easy to register the exit function.

No matter where you crash the program, those functions that you registered will be executed.

The sample is as follows

If the clean() function has arguments, you can skip the decorator and call atexit.register directly (clean_1, argument 1, argument 2, argument 3=’ XXX ‘).

There may be other ways you can handle this requirement, but it’s certainly more elegant and convenient than not using Atexit, and it’s easy to extend.

However, there are still some limitations to using AtExit, such as:

If the program is killed by a system signal that you did not process, the registered function will not execute properly.
If a serious Python internal error occurs, your registered functions will not execute properly.
If you call it manuallyos._exit()The function you registered cannot execute properly.

9. Implement deferred calls like defer

There is a mechanism in Golang to defer calls with the keyword defer, as shown in the following example

import "fmt"

func myfunc(a) {
    fmt.Println("B")}func main(a) {
    defer myfunc()
    fmt.Println("A")}Copy the code

The output is as follows, the myfunc call will complete one step before the function returns, even if you write the myfunc call on the first line of the function, which is called deferred.

A
B
Copy the code

Is there a mechanism for this in Python?

There are, of course, but not quite as simple as Golang.

You can do this in Python using the context manager

import contextlib

def callback(a):
    print('B')

with contextlib.ExitStack() as stack:
    stack.callback(callback)
    print('A')
Copy the code

The output is as follows

A
B
Copy the code

10. How to stream read G large files

Use with… open… You can read data from a file, an operation familiar to all Python developers.

But if you don’t use it properly, it can cause a lot of trouble.

For example, when you use the read function, Python loads the entire contents of the file into memory at once. If the file is 10 gigabytes or more, your computer will consume a lot of memory.

# one time read
with open("big_file.txt"."r") as fp:
    content = fp.read()
Copy the code

For this problem, you might want to use readLine to make a generator that returns line by line.

def read_from_file(filename):
    with open(filename, "r") as fp:
        yield fp.readline()
Copy the code

But if this file is just one line, 10 gigabytes in a row, you’re actually going to read it all at once.

The most elegant solution is to specify that only a fixed size is read at a time when using the read method, such as the following code, which returns only 8KB at a time.

def read_from_file(filename, block_size = 1024 * 8):
    with open(filename, "r") as fp:
        while True:
            chunk = fp.read(block_size)
            if not chunk:
                break

            yield chunk
Copy the code

The above code, the function has no problem, but the code looks like the code is still a bit bloated.

You can optimize your code with partial functions and iter functions

from functools import partial

def read_from_file(filename, block_size = 1024 * 8):
    with open(filename, "r") as fp:
        for chunk in iter(partial(fp.read, block_size), "") :yield chunk
Copy the code

5 years of Python skills, summarized 10 development tips! Net friend: Too practical

1. How do I view source code in the running state?

2. How do I disable the automatic association context?

03. Fastest way to view the package search path

4. Write the nested for loop as a single line

5. How to use print to output logs

6. How to calculate the running time of function quickly

7. Use the built-in caching mechanism to improve efficiency

8. The technique of executing code before program exit

9. Implement deferred calls like defer

10. How to stream read G large files

Related Posts

Easily take you to play with the Linux environment under the date syntax!

Build a Python environment

Kimball dimension Modeling is the most common model in data warehouse construction