In “An Easy to understand Deno Tutorial (Seven Aspects to Start with),” Po explained how to build a simple TCP Echo Server using Deno. This article will use this example to explore how TCP Echo Server works. High energy ahead, please take a deep breath and get ready.
Already started Deno, want to start real combat Deno partners, can read a baoge “great Deno entry and actual combat” this article.
Set up the TCP Echo Server
All right, so without further ado, let’s get down to business. TCP Echo Server = TCP Echo Server = TCP Echo Server
echo_server.ts
const listener = Deno.listen({ port: 8080 });
console.log("listening on 0.0.0.0:8080");
for await (const conn of listener) {
Deno.copy(conn, conn);
} Copy the codefor await… The of statement creates an iterative loop on asynchronous or synchronous iterables, including String, Array, array-like objects (such as Arguments or NodeList), TypedArray, Map, Set and custom asynchronous or synchronous iterables.
for await… The syntax of of is as follows:
for await (variable of iterable) { statement } Copy the code
We then start the TCP Echo Server with the following command:
$ deno run --allow-net ./echo_server.ts
Copy the codeThe important thing to note here is that when running./echo_server.ts, we need to set the –allow-net flag to allow network access. Otherwise, the following error message will appear:
Error: Uncaught PermissionDenied: network access to "0.0.0.0:8080", run again with the --allow-net flag
Copy the codeWhy is that? This is because Deno is a JavaScript/TypeScript runtime that uses a secure environment to execute code by default. After the server runs successfully, we use the nc command to test the function of the server:
$ nc localhost 8080
Let's learn about Deno # sending dataLet's learn the data returned by Deno #Copy the codeNc, short for Netcat, is known as the Swiss Army knife of the Internet. Because it is short and functional, it is designed as a simple, reliable network tool.
The role of NC:
The NC can be used as a server to listen on a specified port in TCP or UDP mode.
For port scanning, the NC can serve as the Client to initiate TCP or UDP connections.
Transfer files between machines or network speed measurement between machines.
Let’s take a look at what happens between starting the TCP Echo Server and connecting to the server using the NC command.
2. Analysis of the TCP Echo Server running process
2.1 Starting the TCP Echo Server
After you run the deno run –allow-net./echo_server.ts command, the following information is displayed:
Listening on 0.0.0.0:8080Copy the codeNetstat: TCP echo Server is listening on port 8080. Netstat: TCP echo server is listening on port 8080.
[root@izuf6ghot555xyn666xm888 23178]# netstat -natp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
TCP 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23178/ denOCopy the codeBy observing the preceding network information, we can find that the CURRENT TCP Echo server is in the LISTEN state, and the PID of the current process is 23178.
In Linux, everything is a file. The /proc file system is a virtual file system that, in the form of file system directories and files, provides an interface to kernel data structures through which various system properties can be viewed and changed.
Here we enter the directory and use the ls – 23178 process l | grep ‘^ d’ command to view the current subdirectory in the directory information:
[root@izuf6ghot555xyn666xm888]# cd /proc/23178
[root@izuf6ghot555xyn666xm888 23178]# ls -l | grep '^d'
dr-xr-xr-x 2 root root 0 attr
dr-x------ 2 root root 0 fd
dr-x------ 2 root root 0 fdinfo
dr-x------ 2 root root 0 map_files dr-xr-xr-x 5 root root 0 net dr-x--x--x 2 root root 0 ns dr-xr-xr-x 4 root root 0 task Copy the codeThe /proc/pid/task and /proc/pid/fd directories are:
2.1.1. / proc/pid/task
This directory contains every thread in the process. The name of each directory is named after the thread ID (TID). The directory structure under each TID is the same as the directory structure under /proc/pid. For attributes shared by all threads, each file in the task/tid subdirectory has the same contents as the corresponding file in /proc/pid. For example, the task/tid/ CWD file in all threads has the same content as the /proc/pid/cwd file in the parent directory, because all threads share a working directory. For different attributes of each thread, the corresponding file under task/ TID has different values.
For our Deno process (23178), we use the ls -al command to view the information in the /proc/23178/task directory:
[root@izuf6ghot555xyn666xm888 task]# ls -al
total 0
dr-xr-xr-x 4 root root 0 .
dr-xr-xr-x 9 root root 0 ..
dr-xr-xr-x 6 root root 0 23178
dr-xr-xr-x 6 root root 0 23179 Copy the codeNext we go to the /proc/23178/task directory to begin analyzing the /proc/pid/fd directory.
2.1.2 / proc/pid/fd
This directory contains every file opened by the current process. Each entry is a file descriptor that is a symbolic link to the actual open address. Where 0 represents standard input, 1 represents standard output, and 2 represents standard error. In a multithreaded program, if the main program exits, this folder will not be accessed.
The file descriptor is formally a non-negative integer. In fact, it is an index value that points to a record table of the process open files maintained by the kernel for each process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In programming, some low-level programming tends to revolve around file descriptors. But the concept of file descriptors tends to apply only to operating systems like UNIX and Linux.
Each Unix process (with the exception of possible daemons) should have three standard POSIX file descriptors, corresponding to three standard streams:
An integer value The name of the Unistd. h symbol constant Stdio.h file stream 0 Standard input STDIN_FILENO stdin 1 Standard output STDOUT_FILENO stdout 2 Standard error STDERR_FILENO stderr
For our Deno process (23178), we use the ls -al command to view the information in the /proc/23178/fd directory:
[root@izuf6ghot555xyn666xm888 fd]# ls -al
total 0
dr-x------ 2 root root 0 .
dr-xr-xr-x 9 root root 0 ..
lrwx------ 1 root root 64 0 -> /dev/pts/0
lrwx------ 1 root root 64 1 -> /dev/pts/0 lrwx------ 1 root root 64 2 -> /dev/pts/0 lrwx------ 1 root root 64 3 -> anon_inode:[eventpoll] lr-x------ 1 root root 64 4 -> pipe:[30180039] l-wx------ 1 root root 64 5 -> pipe:[30180039] lrwx------ 1 root root 64 6 -> /dev/pts/0 lrwx------ 1 root root 64 7 -> /dev/pts/0 lrwx------ 1 root root 64 8 -> socket:[30180040] Copy the codeLooking at the above output, we can see that our Deno process (23178) contains other file descriptors besides the 0-2 file descriptor. Here we focus on file descriptor 8, which, according to the output, represents a Socket. So when was this Socket created? Let’s keep that in mind as we explore the internal creation process later.
Let’s look at the next process, which uses the NC command to connect to our TCP Echo Server.
2.2 Connecting to the TCP Echo Server
Next, we connect to our TCP Echo Server using the nc command described earlier:
[root@izuf6ghot555xyn666xm888 ~]# nc localhost 8080
Copy the codeThen enter Hello Semlinker on the keyboard. At this time, hello Semlinker is automatically displayed on the current command line. To check the current network status, run the netstat command as follows:
[root@izuf6ghot555xyn666xm888 fd]# netstat -natp | grep 8080
TCP 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23178/ denOTCP 00 127.0.0.1:55700 127.0.0.1:8080 ESTABLISHED 23274/ NCTCP 00 127.0.0.1:8080 127.0.0.1:55700 ESTABLISHED 23178/ denOCopy the codeIn line 23274/nc, we can see that nc has ESTABLISHED a TCP connection with our TCP Echo server using port 55700 on the local machine, because the current connection state is ESTABLISHED. Let’s run the ls -al command again to check the /proc/23178/fd directory. The command output is as follows:
[root@izuf6ghot555xyn666xm888 fd]# ls -al
total 0
dr-x------ 2 root root 0 .
dr-xr-xr-x 9 root root 0 ..
lrwx------ 1 root root 64 0 -> /dev/pts/0
lrwx------ 1 root root 64 1 -> /dev/pts/0 lrwx------ 1 root root 64 2 -> /dev/pts/0 lrwx------ 1 root root 64 3 -> anon_inode:[eventpoll] lr-x------ 1 root root 64 4 -> pipe:[30180039] l-wx------ 1 root root 64 5 -> pipe:[30180039] lrwx------ 1 root root 64 6 -> /dev/pts/0 lrwx------ 1 root root 64 7 -> /dev/pts/0 lrwx------ 1 root root 64 8 -> socket:[30180040] lrwx------ 1 root root 64 9 -> socket:[30181765] Copy the codeAfter the nc command is used to connect to the TCP echo server, a new file descriptor is added in the /proc/23178/fd directory, that is, 9 -> socket:[30181765], which is also used to represent a socket.
Ok, so now that we’ve seen the phenomenon, what’s the internal process? To analyze the internal flow of execution, we need to use the Strace command provided by Linux, which is used to track the system calls and signals received while the process is executing.
Use strace to track system calls in the process
In order to better understand the following content, we need to introduce some pre-knowledge, such as Socket, Socket API, user mode and kernel mode.
3.1 File descriptors
In Linux, everything is treated as a file, which can be divided into ordinary files, directory files, link files, and device files. When a process opens an existing file or creates a new file, the kernel returns a file descriptor to the process. A file descriptor is an index created by the kernel to efficiently manage opened files. The file descriptor is used to point to the opened file.
Each file descriptor will correspond to an open file, and different file descriptors will point to the same file. The same file can be opened by different processes or multiple times in the same process. The system maintains a table of file descriptors for each process. The values of this table start at 0, so you will see the same file descriptor in different processes. In this case, the same file descriptor may refer to the same file, or it may refer to different files.
To understand file descriptors, we need to understand three data structures maintained by the kernel.
- Process-level file descriptor table;
- System-level open file descriptor table
- I-node table of the file system.
The following figure shows the relationship between file descriptors, open file handles, and i-Nodes:
(Image from the Internet)
The two processes in the figure have many open file descriptors.
3.2 the Socket
Two programs on a network exchange data through a bidirectional communication connection. One end of the connection is called a socket, so at least one pair of port numbers is needed to establish a network communication connection. Socket is the encapsulation of TCP/IP protocol stack. It provides a programming interface for TCP or UDP, not another protocol. With sockets, you can use the TCP/IP protocol.
The original meaning of Socket is “hole” or “Socket”. As the process communication mechanism of BSD UNIX, the latter meaning is taken. Also known as a “socket”, it describes IP addresses and ports, and is a handle to a communication chain that can be used to communicate between different virtual machines or different computers.
Hosts on the Internet generally run multiple service software and provide several services simultaneously. Each service opens a Socket and binds it to a port. Different ports correspond to different services. A Socket, as the English word means, is like a porous Socket. A mainframe is like a room filled with sockets, each with a number, some with 220 volts ac, others with 110 volts AC, and others with cable TV programs. Customer software plugs plugs into different numbered sockets to get different services. — Baidu Encyclopedia
The following points can be summarized about sockets:
- It can realize the underlying communication, almost all the application layer is through the socket communication.
- It encapsulates TCP/IP protocol to facilitate protocol invocation at the application layer. It is an intermediate abstraction layer between the two.
- In the TCP/IP protocol family, the transport layer has two common protocols: TCP and UDP. The two protocols are different because the socket implementation process of different parameters is different.
The following figure illustrates the client/server relationship for the socket API of the connection-oriented protocol.
3.3 the Socket API
(1) Socket () function: used to create a socket and configure various socket properties, returning a descriptor.
int socket(int af, int type, int protocol);
Copy the code- Af is an Address Family, that is, an IP Address type. Commonly used IP addresses are AF_INET and AF_INET6. AF is short for “Address Family” and INET is short for “Inetnet”. AF_INET indicates an IPv4 address, and AF_INET6 indicates an IPv6 address.
- Type indicates the data transmission mode/socket type. The common types are SOCK_STREAM (stream socket) and SOCK_DGRAM (datagram socket).
- Protocol Indicates the transport protocol. Commonly used are IPPROTO_TCP and IPPTOTO_UDP, which indicate the TCP and UDP transport protocols respectively.
Usage:
int tcp_socket = socket(AF_INET, SOCK_STREAM, 0); // Create the TCP socket
int udp_socket = socket(AF_INET, SOCK_DGRAM, 0); // Create the UDP socket
Copy the code(2) The bind() function is used to bind a socket to a specific IP address and port so that data flowing through the IP address and port can be processed by the socket.
int bind(int sock, struct sockaddr *addr, socklen_t addrlen);
Copy the codeSock is the socket file descriptor, addr is the pointer to the sockaddr structure variable, and addrlen is the sizeof the addr variable, which can be calculated by sizeof().
Usage:
// Create the socket
int listenfd = socket(AF_INET, SOCK_STREAM, 0);
// Create the sockaddr_in structure variable
struct sockaddr_in serv_addr;
memset(&serv_addr, 0.sizeof(serv_addr)); // Each byte is filled with 0
serv_addr.sin_family = AF_INET; // Use the IPv4 address serv_addr.sin_addr.s_addr = inet_addr("127.0.0.1"); // The specific IP address serv_addr.sin_port = htons(8080); / / port // Bind the socket to the IP and port bind(listenfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)); Copy the codeBind the created socket to IP address 127.0.0.1 and port 8080.
(3) The listen() function is used to put the socket into a passive listening state. Passive listening means that the socket is “asleep” when there is no client request, and is only “awakened” to respond to a request when a client request is received.
int listen(int sock, int backlog);
Copy the codeSock indicates the socket that needs to be monitored, and backlog indicates the maximum length of the request queue. If a new request comes in while the socket is processing a client request, the socket cannot handle it. It can only put it into a buffer and then read it out of the buffer after the current request has been processed. If new requests keep coming in, they are queued up in the buffer in order until the buffer is full. This buffer is called a Request Queue.
When the queue is full, no more requests will be received. On Linux, the client will receive the ECONNREFUSED error. On Windows, the client will receive the WSAECONNREFUSED error. Note that the listen() function simply puts the socket in the listening state and does not accept requests. Receiving a request requires the accept() function.
(4) Accept () function: When the socket is in the listening state, the accept() function can be used to receive client requests.
int accept(int sock, struct sockaddr *addr, socklen_t *addrlen);
Copy the codeIts parameters are the same as those of listen() : sock is the server socket, addr is the sockaddr_in structure variable, and addrlen is the length of the parameter addr, which can be obtained by sizeof().
The accept() function returns a new socket to communicate with the client. Addr holds the client’s IP address and port number. Sock is the server’s socket.
Note that listen() simply puts the socket into listening state. It doesn’t actually accept the client request. The code following Listen () continues execution until it encounters accept(). Accept () blocks program execution until a new request arrives. After introducing these core Socket apis, let’s take an example of a Server Socket to give you a better understanding of how these functions are used.
simple_tcp_demo.c
#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <string.h> #define PORT 8080 int main(int argc, char const *argv[]) { int server_fd, new_socket, valread; struct sockaddr_in address; int opt = 1; int addrlen = sizeof(address); char buffer[1024] = {0}; char *hello = "Hello from server"; /* create a listening socket using the IPV4 address */ if ((server_fd = socket(AF_INET, SOCK_STREAM, 0)) = =0) { perror("socket failed"); exit(EXIT_FAILURE); } /* ② Set socket configurations */ if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt))) { perror("setsockopt"); exit(EXIT_FAILURE); } /* AF_INET: IPv4 address used by the Internet, and AF_INET6: IPv6 address used by the Internet */ address.sin_family = AF_INET; /* INADDR_ANY = 0.0.0.0; /* INADDR_ANY = 0.0.0.0;Or "all addresses", "any address". * / address.sin_addr.s_addr = INADDR_ANY; /* The network side always uses Big endian, but the native side depends on the processor, for example, x86 side is different from the network side,I'm going to use Little endian.Htons: Host To Network Short, which converts the endian of the local end ToThe byte order of the network side */ address.sin_port = htons( PORT ); /* ③ Bound to the local address, port 8080 */ if (bind(server_fd, (struct sockaddr *)&address, sizeof(address))<0) { perror("bind failed"); exit(EXIT_FAILURE); } /* to better understand the backlog argument, we must realize that the kernel maintains two queues for any given listener socket:- Incomplete Connection Queue. Each such SYN section corresponds to one of these entries:Has been sent by a client and reached the server, and the server is waiting to complete the corresponding TCP three-way handshake process. These socketsIn the SYN_RCVD state.- Completed Connection Queue for each client that has completed the TCP three-way handshakeIt corresponds to one of these terms. These sockets are in the ESTABLISHED state. * / if (listen(server_fd, 3) < 0) { perror("listen"); exit(EXIT_FAILURE); } /* the accept() function takes a completed connection from the header of the connection queue in the Established state.If the queue has no completed connections, the accept() function blocks until the completed user connections in the queue are retrieved. * / /* While (true) or for (;;) Loop through user requests */ if ((new_socket = accept(server_fd, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0) { perror("accept"); exit(EXIT_FAILURE); } /* Read the data sent by the client */ valread = read( new_socket , buffer, 1024); printf("%s\n",buffer ); /* Return data to client */ send(new_socket , hello , strlen(hello) , 0 ); printf("Hello message sent\n"); return 0; } Copy the codeFor the above simple_tcp_demo.c code, you can compile and run it with GCC:
$ gcc simple_tcp_demo.c -o simple_tcp_demo && ./simple_tcp_demo
Copy the codeThen we continue to use the NC command to connect to the server:
$ nc localhost 8080
hello deno
Hello from server%
Copy the codeIf all is well, you can see the following output on the command line terminal:
$ tcp-server gcc simple_tcp_demo.c -o simple_tcp_demo && ./simple_tcp_demo
hello deno
Hello message sent
Copy the code3.4 User mode and kernel mode
The Linux operating system architecture is divided into user-mode and kernel-mode (or user-space and kernel-space). The kernel is essentially a piece of software — it controls the hardware resources of a computer and provides the environment in which upper-layer applications run. User mode is the activity space of upper-layer applications. The execution of applications must depend on the resources provided by the kernel, including CPU resources, storage resources, and I/O resources.
In order for upper-layer applications to access these resources, the kernel must provide an interface for upper-layer applications to access: a system call.
The minimum functional unit of the operating system at the time of a system call. Depending on the application scenario, the number of system calls provided by different Linux distributions varies from 240 to 350. These system calls form the basic interface for user-mode and kernel-mode interactions. In the actual operating system, in order to shield these complex low-level implementation details and relieve the burden of developers, the operating system provides us with library functions. It implements the encapsulation of system calls, presents simple business logic interfaces to users, and makes it convenient for developers to call.
Let’s use the write() function as an example to demonstrate the system call process:
(photo: https://www.linuxbnb.net/home/adding-a-system-call-to-linux-arm-architecture/)
In addition to system calls, let’s take a quick look at the Shell. Some of you have already written Shell scripts. Shell is a special application, commonly known as the command line, is essentially a command interpreter, under which the system calls, on a variety of applications, often act as a kind of “glue” role, to connect the small program function, make different programs to work together with a clear interface, so as to enhance the function of each program.
In order to facilitate the interaction between users and the system, a Shell corresponds to a terminal, which is a hardware device and presents a graphical window to users. Of course, as we mentioned earlier, the Shell is programmable. It has a standard Shell syntax, and the text that conforms to that syntax is commonly referred to as a Shell script.
Now the question is, how do you switch from user mode to kernel mode? You can switch states in the following three ways:
- System call: the system call itself is interrupt, but soft interrupt, unlike hard interrupt.
- Exception: If the current process is running in user mode, the switch will be triggered if an exception event occurs.
- Peripheral interrupt: When a peripheral completes a user’s request, it sends an interrupt signal to the CPU.
3.5 the strace command
The strace command is used to track system calls and signals received while a process is executing. In the Linux world, processes do not have direct access to hardware devices. When a process needs to access a hardware device (such as reading disk files, receiving network data, and so on), it must switch from user mode to kernel mode and access the hardware device through system calls. Strace tracks system calls generated by a process, including parameters, return values, and execution time.
Next, we’ll use the strace command to track the flow of system calls through the Deno TCP Echo Server process. Start by typing the following command on the command line:
[root@izuf6ghot555xyn666xm888 deno]# strace -ff -o ./echo_server deno run -A ./echo_server.ts
Copy the code-ff: If -o filename is provided, the tracing results of all processes are displayed in filename. Pid is the process ID of each process.
-o filename: writes the output of strace to filename.
After the command is successfully executed, the following two files are generated in the /home/deno directory:
-rw-r--r-- 1 root root 14173 echo_server.23178
-rw-r--r-- 1 root root 137 echo_server.23179
Copy the codeIn order to more intuitive understanding of the 23178 and 23179 these two processes, here we are again through the pstree – ap | grep deno command will deno related process is displayed in the form of a tree:
[root@izuf6ghot555xyn666xm888 deno]# pstree -ap | grep deno
| | `-strace,23176 -ff -o ./echo_server deno run -A ./echo_server.ts
| | `-deno,23178 run -A ./echo_server.ts
| | `-{deno},23179
| |-grep,23285 --color=auto deno
Copy the codeBy observing the above process tree, we can know that the process ID of our TCP Echo Server process is 23178. We can verify our guess by checking the current network status:
[root@izuf6ghot555xyn666xm888 deno]# netstat -natp | grep deno
TCP 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 23178/ denOCopy the code/home/deno/echo_server.23178 /home/deno/echo_server.23178 /home/deno/echo_server.23178
When the TCP Echo Server is started, the socket() function is called to create a listening socket. The socket is then bound to the local address 0.0.0.0 and port 8080, so that the data flowing through the IP address and port can be processed by the socket. The listen() function, such as LISTEN (8, 128), is then called again, putting the socket into a passive listening state.
At this point we go to /proc/23178/fd and use ls-al to check the status of the current directory. Here we see the expected file description — 8 -> socket:[30180040].
[root@izuf6ghot555xyn666xm888 fd]# ls -al
total 0
dr-x------ 2 root root 0 .
dr-xr-xr-x 9 root root 0 ..
lrwx------ 1 root root 64 0 -> /dev/pts/0
lrwx------ 1 root root 64 1 -> /dev/pts/0 lrwx------ 1 root root 64 2 -> /dev/pts/0 lrwx------ 1 root root 64 3 -> anon_inode:[eventpoll] lr-x------ 1 root root 64 4 -> pipe:[30180039] l-wx------ 1 root root 64 5 -> pipe:[30180039] lrwx------ 1 root root 64 6 -> /dev/pts/0 lrwx------ 1 root root 64 7 -> /dev/pts/0 lrwx------ 1 root root 64 8 -> socket:[30180040] Copy the codeNext we use the nc command to connect to our TCP Echo server:
[root@izuf6ghot555xyn666xm888 deno]# nc localhost 8080
Copy the codeAs we already know, a new file descriptor is added to /proc/23178/fd when the connection is successfully created:
lrwx------ 1 root root 64 9 -> socket:[30181765]
Copy the codeAs we’ve already shown, when a socket is in the listening state, the accept() function can be used to receive client requests. In addition, the accept() function returns a new socket to communicate with the client. Let’s go ahead and open /home/deno/echo_server.23178. Here we find something related to accept:
It can be seen from the figure that the socket socket corresponding to the file descriptor 9 is generated after the nc command is invoked. When the client establishes a connection with the server, it returns a new socket to communicate with the client. Some readers have also noticed that in addition to accepT4, there are epoll_CTL and epoll_WAIT functions associated with IO multiplexing.
Epoll is an extensible I/O event notification mechanism of the Linux kernel. Debuting on Linux 2.5.44, it is designed to replace the existing POSIX select and poll system functions and give better performance to programs that require extensive manipulation of file descriptors. The ePoll implementation is similar to poll in that it listens for events on multiple file descriptors.
Epoll is similar to FreeBSD’s KQueue in that the underlying layer is constructed from configurable OS kernel objects that are presented in user space in the form of file descriptors. Epoll searches for monitored file descriptors by using rb-tree.
I’m not going to go into IO multiplexing as it relates to epoll, but I’m going to do an article on IO multiplexing when I have time, and I’m going to talk about the difference between select, poll, and epoll.
Iv. Reference Resources
- Socket () function
- Introduction to the /proc directory in Linux
- Strace traces the system call in the process
- How to understand Linux user mode and kernel mode?
- The relationship between file descriptors and open files in Linux
🏆 technology project phase I | talk about Deno some thing…
This article was typeset using MDNICE