Do you want to ping google.com to determine network connectivity? I think you’re insulting Professor Fong. This is the fifth installment in the Cast Away series, but it won’t teach you fQ. See Synonyms at:

Linux “Cast Away” (a) preparation

Linux “Cast Away” (2) CPU

Cast Away for Linux (3) memory chapter

Cast Away (4) I/O

Looking at hundreds of kali Linux network commands, I was lost in thought. There are so many professional network commands, if I had to list, there are thousands of them. Individuals are not penetration testers and have only scratched the surface of most functions. So this article is a very superficial technical summary, focusing only on some of the Linux commands that are commonly used at work.

Thanks to the popularity of NIO, ck10K problems have become a thing of the past. Any server can now support hundreds of thousands of connections. So let’s figure out how many resources are needed for a million connections.

First, each connection is a file handle, so the number of file descriptors is required to support it. Each socket occupies between 15K and 20K memory, so only maintaining the corresponding socket requires 20G memory. The bandwidth required to broadcast a 1KB message is 1000M!

View the current system connection

How do you see how many connections are currently on the system? You can use Netstat in conjunction with AWK for statistics. The following script counts the number of TCP connections in each state

# netstat -antp | awk '{a[$6]++}END{ for(x in a)print x,a[x]}'
LISTEN 41
CLOSE_WAIT 24
ESTABLISHED 150
Foreign 1
TIME_WAIT 92
Copy the code

But if you run this command on a server with tens of thousands of connections, you may wait a long time. So, we have a second-generation network state statistics tool: netstat => SS (not to be confused with the jailbreak tool).

# ss -s
Total: 191 (kernel 220)
TCP:   5056 (estab 42, closed 5000, orphaned 3, synrecv 0, timewait 5000/0), ports 3469
...
Copy the code

Netstat belongs to the Net-Tools tool set, while SS belongs to iproute. It is time to say Bye to Net-tools.

use net-tools iproute
statistical ifconfig ss
address netstat ip addr
routing route ip route
A neighbor arp ip neigh
VPN iptunnel ip tunnel
VLAN vconfig ip link
multicast ipmaddr ip maddr

Ss command

The basic use

Let’s look at the usage of SS in terms of usage scenarios.

View the TCP connection that the system is listening to

ss -atr 
ss -atn # only IP
Copy the code

View all connections in the system

ss -alt
Copy the code

Check the process PID listening on port 444

ss -ltp | grep 444
Copy the code

Check which ports process 555 occupies

ss -ltp | grep 555
Copy the code

All UDP connections are displayed

ss -u -a
Copy the code

To view TCP Sockets, use the -TA option to view UDP Sockets, use the -UA option to view RAW Sockets, use the -wa option to view UNIX Sockets, use the -XA option

All connections to an IP address

Ss DST 10.66.224.130 SS DST 10.66.224.130: HTTP SS DST 10.66.224.130: SMTP SS DST 10.66.224.130:443Copy the code

Displays all HTTP connections

ss  dport = :http
Copy the code

View the top 10 IP addresses that are connected to the host

netstat -antp | awk '{print $4}' | cut -d ':' -f1 | sort | uniq -c  | sort -n -k1 -r | head -n 10
Copy the code

Recv – Q and Send – Q

Note the result of ss execution. Let’s describe recv-q and send-q.

LISTEN
ESTAB

LISTEN state

  • Recv-q: indicates how many connections have not been accepted. For example, Nginx is slow to accept new connections

  • Send-q: represents the listen backlog value

State of ESTAB

  • Recv-q: How many bytes of data in the kernel remain unread by the application and blocked to some extent

  • Send-q: indicates the number of bytes in the Send queue in the kernel that have not received ack

Viewing Network Traffic

Check the traffic

There are many tools to look at network traffic, but SAR is my favorite. SAR is the most versatile monitoring software available on Linux. As shown, you can refresh network traffic once per second using SAR -n DEV 1.

Of course, you can also use ifstat, nload, iptraf, etc. The data source, however, comes from our /proc directory

watch cat /proc/net/dev
Copy the code

View the IP addresses that occupy the most traffic

Sometimes we find that the bandwidth usage is very high, but we can’t tell where the traffic is coming from. That’s where IFtop comes in. As shown in the figure, it is easy to find out which host traffic is coming from.

caught

tcpdump

If you need to check whether there is traffic or debug a difficult Netty application, you can use packet capture to further determine. On Linux, you can use the tcpdump command to capture data and then use the Wireshark to analyze the data.

tcpdump -i eth0 -nn -s0 -v port 80
Copy the code
  • -iThe specified network adapter captures packets
  • -nThe same as SS, the domain name is not resolved
  • -nnTwo n’s indicate that the port is also a number, otherwise it is resolved to a service name
  • -sSet the packet capture length. 0 indicates no limit
  • -vDetailed output is displayed during packet capture. -vv and -VVV are more detailed in sequence

1) Add the -a option to print ASCII and -x to print hex code.

tcpdump -A -s0 port 80
Copy the code

2) Capture related packets of a specific IP address

Tcpdump -i eth0 host 10.10.1.1 tcpdump -i eth0 DST 10.10.1.20Copy the code

3) The -w parameter writes the captured package to a file

tcpdump -i eth0 -s0 -w test.pcap
Copy the code

4) Tcpdump supports expressions, as well as more complex examples such as fetching get and POST requests (non-HTTPS) from the system

tcpdump -s 0 -v -n -l | egrep -i "POST /|GET /|Host:"
Copy the code

See more at hackertarget.com/tcpdump-exa…

To view the captured data, use the Wireshark.

HTTP caught

The capture tool acts as a proxy, capturing your browser-to-server traffic and providing the ability to modify, replay, and batch execute. Is to find problems, analyze protocols, attack sites. Commonly used are the following three:

  • Burpsuite (cross-platform)
  • Fiddle2 (Win)
  • Charles (Mac)

Do bad things in secret.

Traffic copy

You may need to reproduce HTTP real traffic from your production environment in your development environment or in your preview environment to make use of traffic replication. There are three tools to choose from, and I prefer Gor.

  • Gor
  • TCPReplay
  • TCPCopy

Too many connections

According to TCP/IP, a socket contains about 10 connection states. TIME_WAIT and CLOSE_WAIT are common exceptions, except for SYN denial-of-service attacks. TIME_WAIT is usually resolved by optimizing kernel parameters; CLOSE_WAIT is usually caused by poor programming and should be noticed by developers.

TIME_WAIT

TIME_WAIT is the state held by the party that actively closes the connection. Nginx and crawler servers often have a large number of connections in TIME_WAIT state. After closing the connection, TCP waits for 2MS and then closes the connection completely. Because HTTP uses TCP, there is a huge backlog of TIME_WAIT connections on these frequently-opened servers.

Some systems can see the following information through DMESG.

__ratelimit: 2170 callbacks suppressed
TCP: time wait bucket table overflow
TCP: time wait bucket table overflow
TCP: time wait bucket table overflow
TCP: time wait bucket table overflow
Copy the code

Using the ss-s command, you can see that there are 2W timewaits.

ss -s
Total: 174 (kernel 199)
TCP:   20047 (estab 32, closed 20000, orphaned 4, synrecv 0, timewait 20000/0), ports 10785
Copy the code

The sysctl command can set these parameters. If you want them to take effect after a restart, add them to the /etc/sysctl.conf file.

# Change the threshold
net.ipv4.tcp_max_tw_buckets = 50000 
# enable fast collection of time-wait Sockets in TCP connections
net.ipv4.tcp_tw_reuse = 1
# Enable timewait fast collection. This has to be turned on, it's turned off by default.
net.ipv4.tcp_tw_recycle= 1   
# change the system default TIMEOUT time, default is 60s
net.ipv4.tcp_fin_timeout = 10
Copy the code

To test the parameters, use the sysctl -w net.ipv4.tcp_tw_reuse = 1 command. If it is written to a file, use sysctl -p to take effect.

CLOSE_WAIT

CLOSE_WAIT is usually caused when the peer is actively closed and we have not handled it properly. To put it bluntly, there is a problem written by the program, which belongs to a larger kind of harm.

Let’s take a typical case of CSDN Homophonic Taro.

in

So, the correct way to shut down HttpClient is to use its API: abort().

Other Common Commands

Application software

# resumable download file
wget -c $url
# Download the whole site
wget -r -p -np -k $url
Send network connection (common)
curl -XGET $url
# Transfer file
scp
sftp
# Data image backup
rsync
Copy the code

Detection tools

# Connectivity detection
ping google.com
Route detection to the peer end
tracepath google.com
# domain name detection
dig google.com
nslookup google.com
# Network scanning tool
nmap
# Stress test
iperf
# Omnidirectional monitoring tool (good stuff)
nmon
Copy the code

The configuration tool

Stop a network card
ifdown
Enable a network card
ifup
# Multi-function management tool
ethtool
Copy the code

Pressure test

wrk
ab
webbench
http_load
Copy the code

Multi-purpose tool

# remote login
telnet
ssh
nc
# firewall
iptables -L
Copy the code

At the end

Aside from the basic tools, many of the network commands mentioned in this article are not pre-installed and need to be installed using YUM. Network programming, I think, read the TCP/IP volume 1: Protocols book, and then write a few Netty applications. I found one online here. You don’t have to pay for it. www.52im.net/topic-tcpip…

If you want to go further, you can choose:

  • TCP/IP Volume 1: Protocols
  • UNIX Network Programming
  • The Definitive Netty Guide

We’ve already covered NIO in I/O and won’t cover it in detail here. When you encounter the so-called unpack sticky packet problems, encounter heartbeat and flow limiting problems, and even encountered traffic shaping problems, then prove that you are closer and closer to a professional network programming programmer.