Multithreading technology

This is the fifth day of my participation in Gwen Challenge

1. Multitasking

1.1 The concept of multitasking

Multitasking: to perform more than one task at a time [each task can be interpreted as every job in your life]

1.2 Multitasking in real life

The operating system can run multiple tasks simultaneously. For example, if you’re playing a video game while communicating with your teammates, that’s multitasking

The OPERATING system alternates tasks. Task 1 executes for 0.01 seconds, switch to Task 2, task 2 executes for 0.01 seconds, switch to task 3, and execute for 0.01 seconds…… And so on and so forth. On the surface, each task is executed alternately, but the CPU is so fast that it feels like everything is being executed at the same time.

Single-core CPU is the concurrent execution of multi-task, the true parallel execution of multi-task can only be realized on the multi-core CPU, but because the number of tasks is far more than the number of CPU cores, so, the operating system will automatically schedule a lot of tasks to each core in turn.
While singing rap and playing basketball CAI Xukun

summary

Concurrency: Refers to the number of tasks that are more than the number of CPU cores. Through the various task scheduling algorithms of the operating system, multiple tasks can be executed “together” (in fact, some tasks are always not executed, because the speed of switching tasks is quite fast, it seems to be executed together).
Parallel: Refers to the number of tasks less than or equal to the number of CPU cores, i.e. the tasks are really executed together

2. Use of threads

2.1 The concept of threads

A thread is a branch of program code that executes during program execution. Each running program has at least one thread

2.2 Single-thread execution [Sing, dance]

from time import sleep
def sing() :
    for i in range(3) :print("Singing {}".format(i))

def dance() :
    for i in range(3) :print("Dancing {}".format(i))
  
if __name__ == "__main__":
    sing()
    dance()
Copy the code

The results

Singing 0 singing 1 singing 2 dancing 0 dancing 1 dancing 2Copy the code

3. Multithreading

3.1 Import thread module

import threading
Or use Thread directly
from threading import Thread
Copy the code

3.2 Description of the Thread parameter of the Thread class

"" Thread([group [, target [, name [, args [, kwargs]]]]]) group: indicates the Thread group. Only None can be used. Target: indicates the name of the target task to be executed. Send parameters to a task in tuple mode kwargs: send parameters to a task in dictionary mode Name: specifies the thread name. """
Copy the code

Thread method

Common methods for creating instance objects by Thread:

Start () : Starts the child process instance (creates the child thread)
Join ([timeout]) : Whether or how many seconds to wait for the child thread to finish

3.4 Multi-threaded execution [Singing, dancing]

import threading
import time

# Singing task
def sing() :
    for i in range(3) :print("Singing {}".format(i))
        time.sleep(1)

# Dance Mission
def dance() :
    for i in range(3) :print("Dancing {}".format(i))

if __name__ == "__main__":
    Create a singing thread
    sing_thread = threading.Thread(target=sing)
    Create a dancing thread
    dance_thread = threading.Thread(target=dance)
    # start thread
    sing_thread.start()
    dance_thread.start()
Copy the code

The execution result

Singing 0 is dancing 0 is dancing 1 is dancing 2 is singing 1 is singing 2Copy the code

3.4 Multithreading performs tasks with parameters

import threading
import time

# Singing task
def sing(count) :
    for i in range(count):
        print("Singing {}".format(i))
        time.sleep(1)

# Dance Mission
def dance(count) :
    for i in range(count):
        print("Dancing {}".format(i))

if __name__ == "__main__":
    Create a singing thread
    sing_thread = threading.Thread(target=sing, args=(3)),Create a dancing thread
    dance_thread = threading.Thread(target=dance, kwargs={"count": 3})
    # start thread
    sing_thread.start()
    dance_thread.start()
Copy the code

4. View the list of fetched threads

Threading.current_thread () gets the current thread of executing code
Threading.enumerate () Gets a list of active threads for the current program

import threading
import time

# Singing task
def sing(count) :
    for i in range(count):
        print("Singing {}".format(i))
        time.sleep(1)

# Dance Mission
def dance(count) :
    for i in range(count):
        print("Dancing {}".format(i))

if __name__ == "__main__":
    Get the current thread of executing code (main thread)
    print("Main thread:", threading.current_thread())
    Get the current active thread
    thread_list = threading.enumerate(a)print("Without creating and executing child threads:", thread_list)
    Create a singing thread
    sing_thread = threading.Thread(target=sing, args=(3, ), name="Singing task.")
    Create a dancing thread
    dance_thread = threading.Thread(target=dance, kwargs={"count": 3}, name="Dance mission.")

    thread_list = threading.enumerate(a)print("When child threads are created and not executed:", thread_list)

    # start thread
    sing_thread.start()
    dance_thread.start()

    thread_list = threading.enumerate(a)print("When creating and executing child threads:", thread_list)

Copy the code

The execution result

<_MainThread(MainThread, started 12780)> <_MainThread(MainThread, started 12780)> [<_MainThread(MainThread, started 12780)>] [<_MainThread(MainThread, started 12780)>, <Thread(singing task, started 12896)>, <Thread(dance task, Started 10412)>] Dancing 1 dancing 2 singing 1 singing 2Copy the code

Summary: A thread is not added to the active list until it is started

5. Pay attention to multithreading

The target

Know the thread execution considerations

5.1 Execution between threads is out of order

import threading
import time


def task() :
    time.sleep(1)
    print("Current thread :", threading.current_thread().name)


if __name__ == '__main__':

   for _ in range(5):
       sub_thread = threading.Thread(target=task)
       sub_thread.start()
Copy the code

The execution result

Current Thread: Thread-3Current Thread: Thread-2Current Thread: Thread-4Current Thread: Thread-1Current Thread: Thread-5
Copy the code

5.2 The main thread will wait for all child threads to finish

import threading
import time


Test whether the main thread will wait for the child thread to finish executing before exiting
def show_info() :
    for i in range(5) :print("test:", i)
        time.sleep(0.5)


if __name__ == '__main__':
    sub_thread = threading.Thread(target=show_info)
    sub_thread.start()

    The main thread delay is 1 second
    time.sleep(1)
    print("over")
Copy the code

5.3 Daemon the Main Thread

import threading
import time


Test whether the main thread will wait for the child thread to finish executing before exiting
def show_info() :
    for i in range(5) :print("test:", i)
        time.sleep(0.5)


if __name__ == '__main__':
    Create child thread to daemon main thread
    # daemon=True
    Daemon main thread 1
    sub_thread = threading.Thread(target=show_info, daemon=True)
    After the main thread exits, the child thread destroys the code that is no longer executing the child thread
    Daemon main thread mode 2
    # sub_thread.setDaemon(True)
    sub_thread.start()

    The main thread delay is 1 second
    time.sleep(1)
    print("over")
Copy the code

5.4 summarize

Execution between threads is out of order.
The main thread waits for all child threads to finish, and can be daemon if necessary

6. Customize threads

The target

Know how to customize threads
Know how to use custom threads to perform corresponding tasks

6.1 Custom thread Code

import threading


# Custom thread class
class MyThread(threading.Thread) :
    Get the parameters of the receive task through the constructor
    def __init__(self, info1, info2) :
        Call the parent constructor
        super(MyThread, self).__init__()
        self.info1 = info1
        self.info2 = info2

    # Define custom thread-related tasks
    def test1(self) :
        print(self.info1)

    def test2(self) :
        print(self.info2)

    # Execute tasks using the run method
    def run(self) :
        self.test1()
        self.test2()


Create a custom thread
my_thread = MyThread(1 "test"."The test 2")
# start
my_thread.start()
Copy the code

Execution Result:

test1test2
Copy the code

6.2 summary

Custom threads cannot specify target because all tasks in custom threads are executed in the run method
The start thread calls the start method, not the run method, because you are not using child threads to perform tasks

Multithreaded – Share global variables

1. Multithreading shared global variable code

import threading
import time


Define global variables
my_list = list(a)Write data task
def write_data() :
    for i in range(5):
        my_list.append(i)
        time.sleep(0.1)
    print("write_data:", my_list)


# Read data task
def read_data() :
    print("read_data:", my_list)


if __name__ == '__main__':
    Create a thread to write data to
    write_thread = threading.Thread(target=write_data)
    Create a thread that reads data
    read_thread = threading.Thread(target=read_data)

    write_thread.start()
    # delay
    # time.sleep(1)
    The main thread waits for the writing thread to finish executing
    write_thread.join()
    print("Start reading data.")
    read_thread.start()
Copy the code

Running results:

write_data: [0.1.2.3.4Read_data: [0.1.2.3.4]
Copy the code

7. Sharing global variables

7.1 Multithreading simultaneously operates on global variables

import threading

Define global variables
g_num = 0


# loop through to add 1 to the global variable
def sum_num1() :
    for i in range(1000000) :global g_num
        g_num += 1

    print("sum1:", g_num)


# loop through to add 1 to the global variable
def sum_num2() :
    for i in range(1000000) :global g_num
        g_num += 1
    print("sum2:", g_num)


if __name__ == '__main__':
    # create two threads
    first_thread = threading.Thread(target=sum_num1)
    second_thread = threading.Thread(target=sum_num2)

    # start thread
    first_thread.start()
    # start thread
    second_thread.start()
Copy the code

Running results:

sum1: 1210949
sum2: 1496035
Copy the code

Note:

Multiple threads failed to simultaneously manipulate data on global variables

7.2 Analysis of possible data errors caused by simultaneous operation of global variables by multiple threads

Both first_thread and second_thread add 1 to the global variable g_num(default: 0).

If g_num=0, first_thread gets g_num=0. The system schedules first_thread to sleeping and second_thread to running, and t2 gets g_num=0
Second_thread then increments the resulting value by 1 and assigns g_num such that g_num=1
The system then schedules second_thread to “sleeping” and first_thread to “running”. Thread T1 adds one to the 0 it gained before and assigns g_num.
This results in g_num=1 even though both first_thread and first_thread increment g_num

7.3 Solutions to Global Variable Data Errors

Thread synchronization: Ensures that only one thread can perform global variable synchronization at a time: synchronizes, runs in a predetermined order. You finish, I’ll say it again, like a real walkie-talkie

How to synchronize threads:

Thread wait (JOIN)
The mutex

Code for a thread to wait

import threading

Define global variables
g_num = 0


# loop 1000000 times incrementing the global variable by 1
def sum_num1() :
    for i in range(1000000) :global g_num
        g_num += 1

    print("sum1:", g_num)


# loop 1000000 times incrementing the global variable by 1
def sum_num2() :
    for i in range(1000000) :global g_num
        g_num += 1
    print("sum2:", g_num)


if __name__ == '__main__':
    # create two threads
    first_thread = threading.Thread(target=sum_num1)
    second_thread = threading.Thread(target=sum_num2)

    # start thread
    first_thread.start()
    The main thread waits for the first thread to complete before executing the code, allowing it to execute the second thread
    # Thread synchronization: one task can be completed before another task can be executed. Only one task can be executed at a time
    first_thread.join()
    # start thread
    second_thread.start()
Copy the code

Execution Result:

sum1: 1000000
sum2: 2000000
Copy the code

7.4 the conclusion

Multiple threads operating on the same global variable may cause resource contention data errors
The thread synchronization approach can solve the problem of resource contention data errors, but the multi-task becomes single-task.

8. The mutex

8.1 Concept of mutex

Mutex: Locks shared data so that only one thread can operate on it at a time.

Note:

The thread that has grabbed the lock is executed first. The thread that has not grabbed the lock needs to wait and release the lock after it is used up. Then other waiting threads grab the lock and execute the lock after the other thread grabs the lock.
It’s not up to us to decide which thread gets the lock, it’s up to CPU scheduling.

Thread synchronization can ensure that multiple threads can safely access competing resources. The simplest synchronization mechanism is to introduce mutex.

A mutex introduces a state for a resource: locked/unlocked

When a thread wants to change the shared data, it locks it first. In this case, the resource status is locked, and other threads cannot change it. Other threads cannot lock the resource again until the thread releases the resource, making its state “unlocked.” The mutex ensures that only one thread writes at a time, thus ensuring data correctness in multithreaded situations.

The threading module defines the Lock variable. This variable is essentially a function that can handle locking easily:

# to create lock
mutex = threading.Lock()

# lock
mutex.acquire()

# release
mutex.release()
Copy the code

Note:

If the lock was not previously locked, acquire will not block
If acquire is already locked by another thread before calling acquire to lock the lock, acquire blocks until the lock is unlocked

Use mutex to add 1 million times to the same global variable by two threads

import threading


Define global variables
g_num = 0

Create a global mutex
lock = threading.Lock()


# loop through to add 1 to the global variable
def sum_num1() :
    # locked
    lock.acquire()
    for i in range(1000000) :global g_num
        g_num += 1

    print("sum1:", g_num)
    # releases the lock
    lock.release()


# loop through to add 1 to the global variable
def sum_num2() :
    # locked
    lock.acquire()
    for i in range(1000000) :global g_num
        g_num += 1
    print("sum2:", g_num)
    # releases the lock
    lock.release()


if __name__ == '__main__':
    # create two threads
    first_thread = threading.Thread(target=sum_num1)
    second_thread = threading.Thread(target=sum_num2)
    # start thread
    first_thread.start()
    second_thread.start()

    # hint: With a mutex, we can't decide which thread gets the lock, which thread gets the lock first, and the thread that doesn't get the lock needs to wait
    # Mutex multitask becomes single task instantly, which means only one thread can execute it at a time
Copy the code

Running results:

sum1: 1000000
sum2: 2000000
Copy the code

You can see the final result, which is as expected after the mutex is added.

8.3 Purpose of Using mutex

To ensure that multiple threads access shared data without resource contention and data errors

8.4 Locking and Unlocking Process

When a thread calls acquire() on a lock, the lock enters the “locked” state.

Only one thread can acquire the lock at a time. If another thread attempts to acquire the lock at this point, the thread becomes “blocked”, called “blocked”, until the lock is unlocked when the thread that owns it calls the release() method.

The thread scheduler selects one of the synchronized blocked threads to acquire the lock and causes the thread to enter the running state.

8.5 summarize

Benefits of locking:

Ensures that a critical piece of code can only be executed completely by a single thread from start to finish

Disadvantages of locking:

When multithreaded execution becomes a piece of code that contains locks that can actually only be executed in single-threaded mode, efficiency drops dramatically
Deadlock is easy to occur when locks are not used properly

9. A deadlock

9.1 Concept of deadlock

Deadlock: Waiting for the lock to be released is a deadlock

9.2 Deadlock Example

Once a deadlock occurs, it causes the application to stop responding. Let’s look at an example of a deadlock

import threading
import time

Create a mutex
lock = threading.Lock()


Make sure that only one thread can evaluate at a time
def get_value(index) :

    # locked
    lock.acquire()
    print(threading.current_thread())
    my_list = [3.6.8.1]
    # Determine if subscript release is out of bounds
    if index >= len(my_list):
        print("Subscript out of bounds :", index)
        return
    value = my_list[index]
    print(value)
    time.sleep(0.2)
    # releases the lock
    lock.release()


if __name__ == '__main__':
    # Simulate a large number of threads to perform the value operation
    for i in range(30):
        sub_thread = threading.Thread(target=get_value, args=(i,))
        sub_thread.start()
Copy the code

9.3 Avoiding deadlocks

Release locks in appropriate places

import threading
import time

Create a mutex
lock = threading.Lock()


Make sure that only one thread can evaluate at a time
def get_value(index) :

    # locked
    lock.acquire()
    print(threading.current_thread())
    my_list = [3.6.8.1]
    if index >= len(my_list):
        print("Subscript out of bounds :", index)
        # release the lock when the subscript is out of bounds, leaving the value available for subsequent threads
        lock.release()
        return
    value = my_list[index]
    print(value)
    time.sleep(0.2)
    # releases the lock
    lock.release()


if __name__ == '__main__':
    # Simulate a large number of threads to perform the value operation
    for i in range(30):
        sub_thread = threading.Thread(target=get_value, args=(i,))
        sub_thread.start()
Copy the code

9.4 summary

Be aware of deadlocks when using mutex, and release the locks where appropriate
Once a deadlock occurs, it causes the application to stop responding

conclusion

After one or two years of study, I haven’t written a blog formally. Self-study has been accumulated by taking notes. Recently, I re-learned Python multi-threading, and I am thinking of writing a blog to share this part of the content seriously over the weekend.

The article is long, give a big thumbs up to those who see it! Due to the author’s limited level, the article will inevitably have mistakes, welcome friends feedback correction.

If you find the article helpful, please like it, comment on it and forward it

Your support is my biggest motivation!!

Python multithreading (in detail)

Multithreading technology

1. Multitasking

1.1 The concept of multitasking

1.2 Multitasking in real life

summary

2. Use of threads

2.1 The concept of threads

2.2 Single-thread execution [Sing, dance]

3. Multithreading

3.1 Import thread module

3.2 Description of the Thread parameter of the Thread class

Thread method

3.4 Multi-threaded execution [Singing, dancing]

3.4 Multithreading performs tasks with parameters

4. View the list of fetched threads

5. Pay attention to multithreading

The target

5.1 Execution between threads is out of order

5.2 The main thread will wait for all child threads to finish

5.3 Daemon the Main Thread

5.4 summarize

6. Customize threads

The target

6.1 Custom thread Code

6.2 summary

Multithreaded – Share global variables

1. Multithreading shared global variable code

7. Sharing global variables

7.1 Multithreading simultaneously operates on global variables

7.2 Analysis of possible data errors caused by simultaneous operation of global variables by multiple threads

7.3 Solutions to Global Variable Data Errors

7.4 the conclusion

8. The mutex

8.1 Concept of mutex

Note:

Use mutex to add 1 million times to the same global variable by two threads

8.3 Purpose of Using mutex

8.4 Locking and Unlocking Process

8.5 summarize

9. A deadlock

9.1 Concept of deadlock

9.2 Deadlock Example

9.3 Avoiding deadlocks

9.4 summary

conclusion

Related Posts

Java-based Aviator expression learning record (21)

Dubbo service routing source code read

Linkerd 2.10(Step by Step) — Elegant Pod off