In doubt

RQ involves a part of the OS call code, involving process and signal processing, the following code caught my attention

def fork_and_perform_job(self, job): child_pid = os.fork() if child_pid == 0: self.main_work_horse(job) else: self._horse_pid = child_pid self.procline('Forked %d at %d' % (child_pid, time.time())) while True: try: os.waitpid(child_pid, 0) break except OSError as e: # In case we encountered an OSError due to EINTR (which is # caused by a SIGINT or SIGTERM signal during # os.waitpid()), we simply ignore it and enter the next # iteration of the loop, waiting for the child to end. In # any other case, this is some other unexpected OS error, # which we don't want to catch, so we re-raise those ones. if e.errno ! = errno.EINTR: raiseCopy the code

RQ forks a child process for each job. The parent process calls WaitPID to wait for the child process to finish. The key point in the code is the comment under except, which generates an OSError when a SIGINT signal is generated, which makes me puzzled.

Signal to review

A signal is a form of communication between processes. Compared with other forms of communication between processes (such as pipes, shared memory, etc.), the information transmitted by a signal is a coarse integer. But because less information is transmitted, signals are easier to manage and use.

The signal module is used in Python to handle signal-related operations, as defined below:

signal.signal(signalnum, handler)
signal.getsignal(signalnum)
Copy the code

Signalnum is a signal, and handler is the signal processing function. Processes can ignore signals, take default actions, or customize actions. When handler is signal.SIG_IGN, the signal is ignored. When handler is singal.sig_dfl, the process takes the default action (default). When handler is a function name, the process takes the actions defined in the function.

  • Send a signal
os.kill(pid, sid)
os.killpg(pgid, sid)
Copy the code
  • Timing signalling
signal.alarm(2)
Copy the code

Note: When using signals in multithreaded environments, only the main thread can set the signal handler and only the main thread can receive the signal.

  • Signal processing practice
import signal import time def register_signal_handlers(): i=0 def sigint_handler(signum, frame): nonlocal i if i>=1: exit() i+=1 print("CTRL C") print(frame.f_code) j=0 def alarm_handler(signum,frame): nonlocal j if j>=2: return else: SIGINT (sig.sigint,sigint_handler) def main(): register_signal_handlers() signal.alarm(2) func=signal.getsignal(signal.SIGINT) while True: time.sleep(2) if __name__=="__main__": main()Copy the code

Signal extension

Python signal handlers do not execute in low-level (C) signal handlers. Instead, the low-level signal handler sets a flag that tells the Virtual Machine to execute the corresponding Python signal handler later (for example, the next Bytecode instruction). Python signal handlers will always execute in the main thread of the main Python interpreter, even if the signal is received in another thread. This means that signals cannot be used as a means of communication between threads.

EINTR

  • Slow call with EINTR: If the process is blocked in a slow system call and the system call is Interrupted when a signal is caught and the corresponding signal handler returns, the call returns with an error and sets errno to EINTR (the corresponding error is described as Interrupted System Call). A permanently blocked system call is one that never returns, and most network support functions fall into this category. For example, if no client is connected to the server, the server’s accept call will always block.

  • Handling method: Manually restart the interrupted system call or ignore the signal.

  • EINTR: Exception InterruptedError is raised when the system call is interrupted by an input signal. Corresponds to errno EINTR. Changed in Python3.5: When a system call is interrupted by a signal, Python now retries the system call unless the signal handler raises another exception (see PEP 475 for the principle) instead of raising InterruptedError.

  • Pep475: System call wrappers provided in the standard library should automatically retry when using EINTR fails to relieve application code of the burden of doing so. If the signal handler returns successfully, the Python wrapper automatically retries the system call, which is much cleaner to write after Python 3.5.

    ## python2 while True: try: data = file.read(size) break except InterruptedError: continue ## python3.5 while True: data = file.read(size)Copy the code

A link to the

  • Signal: juejin. Cn/post / 684490…

  • EINTR:blog.csdn.net/junlon2006/…

  • Retry system calls failing with EINTR:www.python.org/dev/peps/pe…