Tell me about the IPC

Basic concepts of IPC

IPC stands for inter-process Communication. Each have their own different user address space, any process of the global variable in another process all could not see, so want to exchange data between processes must by the kernel, open up a buffer in the kernel, process 1’ll copy data from user space to the kernel buffer, process 2 go read the data from the kernel buffer.

The general process is as shown in the picture. IPC can probably be implemented in several ways, and we will expand on the advantages and disadvantages and features of them.

The pipe

Features:

  • The pipe is half duplex, so the data can only flow in one direction; When two parties need to communicate, two pipes need to be set up.
  • Can only be used between parent and child processes or between sibling processes (processes that are related).
  • Constitute a separate file system: a pipe is a file for the processes on either side of the pipe. But it is different from ordinary files, it does not belong to a file system, but independent portal, a separate file system, and only exist in memory.
  • Data reading and writing: What one process writes to a pipe is read by a process at the other end of the pipe. The written content is added to the end of the pipe buffer each time, and the data is read from the buffer’s head each time.

It looks something like this.

Limitations:

  • Only one-way data flows are supported.
  • Can only be used between related processes.
  • No name.
  • The buffer of the pipe is finite (the pipe system exists in memory, and a page size is allocated to the buffer when the pipe is created).
  • The pipe carries a stream of bytes without format, which requires the reader and writer of the pipe to agree in advance on the format of the data, such as how many bytes count as a message (or command, or record), and so on.

In order to solve the problem of nameless pipes, named pipes are derived. Named pipes differ from anonymous pipes in that they provide a pathname associated with them and exist in the file system as a named pipe file. In this way, even processes that are not related to the creator of a named pipe can communicate with each other through a named pipe as long as they have access to the path. Thus, unrelated processes can exchange data through named pipes. Value of note is that named pipes strictly follow fifO.

Essence: The essence of a pipe is a kernel buffer, from which processes read and write data in a first-in, first-out manner. Processes at one end of the pipe write data sequentially to the buffer, and processes at the other end read data sequentially.

The buffer can be regarded as a circular queue, where the read and write positions grow automatically and cannot be changed arbitrarily. A data can only be read once, and then it will not exist in the buffer.

When the buffer is empty or full, there are certain rules to control the corresponding read or write process into the wait queue. When new data is written to the empty buffer or data is read from the full buffer, the process in the wait queue will be woken up to continue reading and writing.

It looks a lot like a Queue in Java.

signal

Features:

  • Signals are a mechanism used by processes in Linux to communicate or operate with each other. Signals can be sent to a process at any time without knowing the state of the process.
  • If the process is not currently executing, the signal is stored by the kernel until the process resumes execution and is passed to it.
  • If a signal is set to block by a process, the signal is delayed until the blocking is canceled.

The general process is like this.

Life cycle:

  • A signal is generated by a process, and the object to which the signal is transmitted (typically the PID of the corresponding process) is set, and then transmitted to the operating system.
  • Operating system according to the setting of the receiving process (whether blocking) and selective is sent to the receiver, if the receiver blocking the signal (and the signal can be blocked), the operating system will be temporarily keep the signal and not passed, until the process has been relieved of this signal block (if corresponding process has quit, it discards the signal), if the corresponding process without obstruction, The operating system will relay this signal.
  • Purpose process receives the signal after, will give you the signal set according to the current process of pretreatment, suspended the execution of the current code, protect context (including temporary register data, the current position and the current state of the CPU), to implement the interrupt service program, after completion of execution in response to the location of the interrupt. Of course, with a preemptive kernel, a new schedule is also triggered when the interrupt returns.

The message queue

Features:

  • Message queues are linked lists of messages stored in the kernel, and each message queue is represented by a message queue identifier.
  • And pipeline (anonymous pipe: only exists in memory of the file | a named pipe: exist in the actual disk media or file system) of the message queue in the kernel, only in the kernel to restart or to delete a message queue, the message queue can be really deleted.
  • Also, unlike pipes, message queues do not require another process to wait on a queue for messages to arrive before one process writes to the queue.

Contrast:

  • Message carrying: Message queue overcomes the deficiency of signal carrying less information, pipe carrying only unformatted byte stream and buffer size limitation.
  • Message reading: Message queuing enables random query of messages. Messages may not be read in first-in, first-out order, but may be read by message type. More advantages than FIFO.

Classification:

  • POSIX message queues.
  • System V message queue, which is widely used nowadays.

The Shared memory

Features:

  • Enables multiple processes to read and write directly to the same memory space, and is the fastest form of IPC available. It is designed for the low efficiency of other communication mechanisms.
  • To exchange information between multiple processes, the kernel sets aside a memory area that can be mapped to its own private address space by the process that needs to access it. Processes can read and write directly to the memory without copying data, thus greatly improving efficiency.
  • Because multiple processes share a memory segment, some synchronization mechanism (such as semaphore) is required to achieve synchronization and mutual exclusion between processes.

For example, the oft-mentioned Mmap is a way to share memory.

The socket

The characteristics of a socket are determined by three attributes: domain, port number, and protocol type.

Domain:

  1. AF_INET: It refers to the Internet. When a client uses a socket to connect across a network, it needs the IP address and port of the server computer to specify a particular service on a networked machine. So before using the socket as the endpoint of communication, the server application must bind a port before starting communication, and the server waits for a connection from the client at the specified port.
  2. AF_UNIX: represents the UNIX file system, which is the file input/output and whose address is the file name.

Port number: Each TCP/ IP-based process is assigned a unique port and port number. A port is an information buffer that holds input/output information in the Socket. The port number is a 16-bit unsigned integer to distinguish each program on the host (a port number is like a room number in a house), and any port number below 256 is reserved for standard applications. Each socket is combined with an IP address and port so that the whole can be distinguished from each socket.

Protocol Type:

  • Stream socket: is inAF_INETThe domain is implemented over TCP/IP connections and is alsoAF_UNIXThe socket type commonly used in. Streaming sockets provide an orderly, reliable, bidirectional byte stream connection, so the data sent can be guaranteed not to arrive lost, duplicated, or out of order, and it also has a mechanism for resending if errors occur.
  • Datagram sockets: They do not need to establish and maintain a connection, they are usually implemented in the domain over UDP/IP protocols. It has limits on the length of data that can be sent, and the datagram is transmitted as a single network message. It can be lost, copied, or arrived incorrectly. UDP is not a reliable protocol, but it is relatively fast because it does not always need to establish and maintain a connection.
  • Raw socket: Raw socket allows direct access to lower-level protocols, such as IP and ICMP. It is often used to verify the new protocol implementation, or access the existing service configuration of the new device, because the original socket can freely control a variety of protocols under Windows, can control the transmission mechanism at the bottom of the network, so you can use the original socket to manipulate the network layer and transport layer applications. For example, we can use the raw socket to receive ICMP, IGMP packets sent to the local machine, or receive IP packets that cannot be processed by TCP/IP stack, and can also be used to send some self-defined header or self-defined protocol IP packets. Network monitoring technology relies heavily onSOCKET_RAW.

Raw sockets and standard sockets: Raw sockets can read and write IP packets that are not processed by the kernel, while stream sockets can read ONLY TCP data and datagram sockets can read only UDP data. Therefore, if you want to access other protocols to send data, you must use the raw socket.

A semaphore

A semaphore is a counter used by multiple processes to access shared data. The purpose of a semaphore is synchronization between processes (or threads). To obtain a shared resource, the process needs to do the following:

  • Create a semaphore: This requires the caller to specify an initial value, which is usually 1 or 0 for a binary semaphore.
  • Wait for a semaphore: This operation tests the value of the semaphore and blocks if it is less than 0. Also called the P (through) operation.
  • Hang a semaphore: This operation increments the value of the semaphore by 1. Also known as the V (release) operation.

Interprocess synchronization:

Interthread synchronization:

As I mentioned earlier when talking about locks in Go, the underlying lock in Go is implemented by a semaphore.