In my previous article on the concept of Iterable, Iterator, and Generator in Python, I learned that a Generator can be defined in one of two ways:

  • List generator
  • useyieldDefined function

In earlier versions of Python, Coroutines were also implemented through generators, known as generator-based Coroutines. At the end of the previous article on generators, I gave a producer-consumer example that is implemented based on a generator coroutine.

def producer(c):
    n = 0
    while n < 5:
        n += 1
        print('producer {}'.format(n))
        r = c.send(n)
        print('consumer return {}'.format(r))


def consumer(a):
    r = ' '
    while True:
        n = yield r
        if not n:
            return
        print('consumer {} '.format(n))
        r = 'ok'


if __name__ == '__main__':
    c = consumer()
    next(c)  # start consumer
    producer(c)
Copy the code

Looking at this code, I believe that many beginners like me are having a hard time with generator-based coroutine implementations and writing their own coroutine code according to the business. Python implementers also notice this problem because it is so non-Pythonic. Generator-based coroutines will also be deprecated, so this article focuses on the use of the Asyncio package and some of the related class concepts involved. Note: THE Python environment I use is 3.7.

What is a Coroutine?

Coroutines are executed in threads, known as microthreads, but the switching of coroutines is context-free and lighter than threads. A coroutine can interrupt itself at any time to allow another coroutine to start, or it can resume execution from a break, and the scheduling between them is controlled by the programmer (see the producer-consumer code at the beginning of this article).

Define a coroutine

The aysnc and await keywords have been added in Python3.5+, which make it very easy to define and use coroutines. An async declaration is used to define a coroutine in function definition.

import asyncio

# define a simple coroutine
async def simple_async(a):
    print('hello')
    await asyncio.sleep(1) Sleep for 1 second
    print('python')
    
Run a coroutine using asynio's run method
asyncio.run(simple_async())

The result is
# hello
# python
Copy the code

Use await in coroutine if you want to call another coroutine. Note that the await keyword is used in async defined functions, whereas async functions can have no await

# define a simple coroutine
async def simple_async(a):
    print('hello')
    
asyncio.run(simple_async())

# Execution result
# hello
Copy the code

Asyncio.run () runs the incoming coroutine that manages the asyncio event loop. In addition to the run() method, which executes coroutines directly, you can also use the event loop loop

async def do_something(index):
    print(f'start {time.strftime("%X")}', index)
    await asyncio.sleep(1)
    print(f'finished at {time.strftime("%X")}', index)


def test_do_something(a):
    The generator produces multiple coroutine objects
    task = [do_something(i) for i in range(5)]

    Get an event loop object
    loop = asyncio.get_event_loop()
    Execute the task list in the event loop
    loop.run_until_complete(asyncio.wait(task))
    loop.close()

test_do_something()

Run result
# start 00:04:03 3
# start 00:04:03 4
# start 00:04:03 1
# start 00:04:03 2
# start 00:04:03 0
# finished at 00:04:04 3
# finished at 00:04:04 4
# finished at 00:04:04 1
# finished at 00:04:04 2
# finished at 00:04:04 0
Copy the code

You can see that all coroutines are started almost at the same time. Asyncio.run () encapsulates the loop object and its call. Asyncio.run () creates a new event loop object each time to execute the coroutine.

0 x01 Awaitable object

Awaitable objects in Python are corountine, Task, and Future. That is, these objects can be called with the await keyword

await awaitable_object
Copy the code
1. Coroutines (Coroutine)

Coroutines are defined by async def declarations, and one coroutine can be called by another with await

async def nested(a):
    print('in nested func')
    return 13


async def outer(a):

    The coroutine object returned by a coroutine function will not be executed until the await keyword is used
    print(await nested())

asyncio.run(outer())

# Execution result
# in nested func
# 13
Copy the code

If nested() is called directly in the outer() method without await, a RuntimeWarning will be raised

async def outer(a):
    A direct call to the coroutine function does not execute, but returns a coroutine object
    nested()
    
asyncio.run(outer())
Copy the code

Run the program and the console will output the following information

RuntimeWarning: coroutine 'nested' was never awaited
  nested()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Copy the code
2. The Task (Task)

Tasks can be used to execute coroutines concurrently. You can use asyncio.create_task() to encapsulate a coroutine object into a task that will soon be queued and executed.

async def nested(a):
    print('in nested func')
    return 13

async def create_task(a):
    When a coroutine object is packaged into a task, the coroutine is automatically scheduled to run
    task = asyncio.create_task(nested())
    If you want to see the result of task execution
    We can use await to wait for the coroutine to complete and return the result
    ret = await task
    print(f'nested return {ret}')

asyncio.run(create_task())

Run result
# in nested func
# nested return 13
Copy the code

Note: More on concurrency below.

3. Future

A Future is a special low-level object that is the eventual result of an asynchronous operation. When a Future object is waited on, it means that the coroutine will wait until the Future object is finished operating on somewhere else.

Typically, Future objects are not created directly in the application layer code. This object is used in some libraries and asyncio modules.

async def used_future_func(a):
    await function_that_returns_a_future_object()
Copy the code

0 x02 concurrent

1. Task

We know that tasks can be executed concurrently. Asyncio.create_task () is a method that wraps coroutines into tasks.

async def do_after(what, delay):
    await asyncio.sleep(delay)
    print(what)

Create parallel tasks with asyncio.create_task
async def corun(a):
    task1 = asyncio.create_task(do_after('hello'.1)) # Simulate a 1-second task
    task2 = asyncio.create_task(do_after('python'.2)) # Simulate a 2-second task

    print(f'started at {time.strftime("%X")}')
    # Wait for both tasks to complete, both tasks are parallel, so total time the maximum execution time of the two tasks
    await task1
    await task2

    print(f'finished at {time.strftime("%X")}')

asyncio.run(corun())

Run result
# started at 23:41:08
# hello
# python
# finished at 23:41:10
Copy the code

Task1 is a one-second task, and task2 is a two-second task. Both tasks are executed concurrently, consuming a total of two seconds.

2. gather

In addition to using asyncio.create_task(), you can also use asyncio.Gather (), which receives a list of coroutine arguments

async def do_after(what, delay):
    await asyncio.sleep(delay)
    print(what)
    
async def gather(a):
    print(f'started at {time.strftime("%X")}')
    # Use Gather to pass in multiple coroutines
    await asyncio.gather(
        do_after('hello'.1),
        do_after('python'.2),
    )
    print(f'finished at {time.strftime("%X")}')

asyncio.run(gather())

Run result
# started at 23:47:50
# hello
# python
# finished at 23:47:52
Copy the code

The time consumed by the two tasks is the longest among them.

0 x03 reference

  1. Docs.python.org/3/library/a…