A lifelong learner, practitioner, and sharer committed to the path of technology, an original blogger who is busy and sometimes lazy, and a teenager who is occasionally boring and sometimes humorous.

Welcome to search “Jge’s IT Journey” on wechat!

1. What is Squid?

Squid: Squid is a high-performance proxy cache server that supports FTP, Gopher, HTTPS, and HTTP protocols. Unlike most proxy cache software, squids use a single, non-modular, I/O driven process to process all client requests. As application layer proxy service software, SQUids mainly provide cache acceleration and application layer filter control functions.

Why Squid?

Squid is one of the items that came out of the initial content distribution and caching effort. It has grown to include additional features such as powerful access control, authorization, logging, content distribution/replication, traffic management and shaping, to name a few. There are many old and new solutions for dealing with incomplete and incorrect HTTP implementations.

It is mainly divided into the following three points:

1) For ISP: save bandwidth and improve user experience

Squids allow Internet providers to save bandwidth by caching content, where cached content means that data is provided locally and can be seen by users through frequently used content at faster download speeds. A well-tuned proxy server, even without caching, can improve user speed by optimizing TCP streams. The server is easy to tune to handle the kinds of latency on the Internet that the desktop environment is simply not suited to.

Squids enable ISPs to cope with the growing demand for content without having to spend a lot of money upgrading their core equipment and transmission links, and also allow ISPs to prioritize and control certain Web content types for technical or economic reasons.

2) For websites: Extend the application without spending a lot of hardware and development time

Many web sites use squids to reduce server load, and the usual content is cached by squids and supplied to clients, often requiring only a small fraction of the application server load. Setting up an accelerator in front of an existing site is almost always a quick and easy task, and has immediate benefits.

3) For content delivery providers: distribute your content globally

Squids make it easy for content distributors and developers to distribute content around the world. CDN providers can buy inexpensive PC hardware running SQUIds and then deploy it in strategic locations on the Internet to deliver large amounts of data cheaply and efficiently.

The working mechanism of Squid agent

When a client requests a Web page through a proxy, the specified proxy server will first check its own cache. If there are pages in the cache that the client needs, the specified proxy server will directly feed back the page content in the cache to the client.

If there is no page in the cache that the client wants to access, the proxy server sends an access request to the Internet. When the returned Web page is obtained, the Web page data is saved in the cache and sent to the client.

The brief description flow chart is as follows:

Cache accelerated objects in HTTP proxies are mainly static Web elements such as text, images, and so on. Using caching mechanism, when the client at the time of different access to the same page, or different client to access the same web page, can be obtained directly from the proxy cache results, at the same time also greatly reduces the submit a repetitive process of web page request to the Internet, can enhance the response speed of the client’s web access.

The client’s web page access request is completed by proxy server to replace, can hide the user’s real IP address, and play a certain role in protection. At the same time, it also filters the target to access, the address of the client, the access time period, and so on.

Basic types of Squid agents

  • Traditional proxy: also referred to as the common proxy service. You need to manually set the IP address and port number of the proxy server in the browser and other programs on the client to use the proxy to access the network. The browser sends domain name resolution requests to the specified proxy server when accessing websites.

  • Transparent proxy: Provides the same functions and services as the traditional proxy. The difference is that the client does not need to specify the address and port of the proxy server. Instead, the client uses the default route and firewall policy to redirect the access to the web page.

Five, Squid source package

In Linux, Squid RPM Packages are included in Packages. It’s also the easiest way to get up and running squids quickly, and a great way to keep the supported versions of Squids up to date. In special cases, Squid binary software packages can also be downloaded for a variety of platforms, including Windows;

Address:

Wiki.squid-cache.org/SquidFaq/Bi…

Source package, download can be customized Squid installation at compile time. Latest release: SQUID-4.10.

Squid-4.10 Download address:

www.squid-cache.org/Versions/v4…

You can also see Compiling Squid for help compiling source code.

Address:

Wiki.squid-cache.org/SquidFaq/Co…

For more software package versions, visit the official website.

Address: www.squid-cache.org/Versions/

Set up Squid proxy server

First, set up the environment and equipment, configure the IP address.

1) One Client (Windows Server/Windows) 2) One Web Server (HTTPD service enabled) 3) One Squid proxy Server (two network adapters must be configured: one for the internal IP address and the other for the public IP address)

Client Configuration

First, set the IP address of the client to 192.168.3.1 (Intranet IP address) and network adapter to VMnet8. Then check whether the IP address is correctly configured and the firewall is correctly disabled.

Web server Configuration

SElinux # setenforce 0 # RPM -q HTTPD // Check whether the HTTPD RPM package exists in the system Httpd-2.2.15-29.el6.centos.x86_64 # service HTTPD start # echo "Welcome Jack to this website." > /var/ WWW/HTML /index.html // Welcome Jack to this website. Write to # curl http://202.100.0.100 // to test whether the Welcome Jack to this website output is normal.Copy the code

Squid Proxy server configuration

Decompress the Squid source package, configure Squid compilation options, and set the installation directory to /usr/local/squid, other specific options can be configured according to actual needs, or refer to the./configure –help description.

/configure --prefix=/usr/local/squid --sysconfdir=/etc --enable-arp-acl --enable-linux-netfilter --enable-linxu-tproxy --enable-async-io=100 --enable-poll --enable-err-language="Simplify_Chinese" --enable-undersxcore --enable-poll --enable-gnuregex # make && make installCopy the code

Meanings of the above compilation options:

  • –prefix=/usr/local/squid: installation directory

  • –sysconfdir=/etc: modify the configuration file separately to another directory

  • –enable-arp-acl: you can set the rule to be managed directly by the client MAC address to prevent IP spoofing on the client

  • –enable-linux-netfilter: Uses kernel filtering

  • –enable-linxu-tproxy: supports the transparent mode

  • –enable-async-io=100: indicates asynchronous I/O to improve storage performance

  • –enable-poll: Uses the poll() mode to improve performance

  • –enable-err-language=”Simplify_Chinese” : indicates the language in which error information is displayed

  • –enable-undersxcore: Allows underscores in urls

  • –enable-gnuregex: Uses GNU regular expressions

Create link files, users, and groups
# ln -s /usr/local/squid/sbin/* /usr/local/sbin/
# useradd -M -s /sbin/nologin squid
# chown -R squid:squid /usr/local/squid/var
Copy the code
Modify the Squid configuration file

The Squid service configuration file is located in /etc/squid.conf. Understanding the configuration line helps administrators flexibly configure the agent service according to actual conditions.

# vim /etc/squid.conf 59 http_port 3128 // specifies the address and port that the proxy service listens to. 3128 60 cache_effective_USER SQUID // Specifies the program user of squid, which is used to set the account that is cached during initialization and runtime. Otherwise, the system fails to start. 61 CACHE_effective_group SQUID // The default value is cache_effective_user. 76 visIBLE_HOSTNAME squid.packet-pushers.net // Add the visIBLE_hostname configuration in the last line of the configuration file. Otherwise, the Squid service cannot be started.Copy the code
Squid, check if the configuration file syntax is correct;
# squid -k parse
Copy the code
Start and stop Squid

The first time you start the Squid service, the cache directory is automatically initialized. In the absence of a Squid service script available, the Squid program can be called to start the service, first initialized.

# squid-z // The -z option initializes the cache directory # squid // Starts the SQUID serviceCopy the code
Check whether the Squid service is listening to port 3128;
# netstat -anpt | grep "squid"
tcp        0      0 :::3128                     :::*                        LISTEN      19068/(squid-1)
Copy the code

In addition to installing the Squid service using source code packages, you can also write Squid service scripts and manage them using chkconfig and service tools. In this way, you can start, stop, restart the Squid service using Squid service scripts. During execution, Add the corresponding parameters.

#! /bin/bash # chkconfig: 2345 90 25 # config: /etc/squid.conf # pidfile: /usr/local/squid/var/run/squid.pid # Description: Squids script PID = "/ usr/local/Squid/var/run/squids. PID" CONF = "/ etc/Squid. CONF" CMD = "/ usr/local/Squid/sbin/squids" case "$1" in Start) netstat anpt | grep squids & > / dev/null if [$? - eq 0] then echo "squids is running" else echo "squid are launching..." $CMD fi ;; stop) $CMD -k kill &> /dev/null rm -rf $PID &> /dev/null ;; $0 stop &> dev/null echo "Squid..." $0 start &> /dev/null echo "Squid..." ;; reload) $CMD -k reconfigure ;; check) $CMD -k parse ;; Status)/f $PID & > / dev/null if [$? - eq 0] then netstat anpt | grep squids else echo squid "no running" fi;; *) echo "usage $0 {start | stop | restart | check | status}";; Esac # chmod +x /etc/init.d/squid # chkconfig --add squid // add squid to system service # chkconfig squid onCopy the code

Vii. Build a proxy server

Traditional proxy: Mainly lies in the client related programs, must specify the proxy server address, port and other information.

Requirement description:

1) Provide proxy service for clients to access various websites on Squid proxy server, but prohibit downloading files over 10MB by proxy;

2) Specify Squid proxy server as Web access proxy on the client, and hide the real IP address of the client.

As a Squid proxy server, the Squid service must be set up and the client must be allowed to use the proxy. When the client through the proxy in the form of IP address to access. The client must specify the proxy server address and port number for the browser and enable the HTTPD service on the Web server.

Squid proxy server configuration

When configuring Squid to implement the traditional proxy service, you need to add http_access Allow all to allow any client to use the proxy service and limit the download file size. You need to configure the REPLy_body_max_size option.

Modify the squid. conf configuration file

# vim /etc/squid.conf 59 http_port 3128 60 reply_body_max_size 10 MB Before http_access allow allCopy the code

Add a permit policy to the firewall and save the policy

# iptables -I -INPUT -p tcp --dport 3128 -j ACCEPT
# service iptables save
Copy the code

Restart the Squid service

# service squid reload
Copy the code

Proxy configuration for the client

In the Browser, select tools – Internet Options, the Internet Options dialog box is displayed, in the Connection TAB, click the LAN Settings button, to configure the IP address and port of the proxy server.

The authentication method for the proxy service

In the client 192.168.3.100 through the browser to access the target website http://202.100.0.100/, and through the Squid proxy server, Web site server access log, to verify whether the proxy service plays a role.

View the Squid access log record

On the Squid proxy server, by tracing the access log file of the Squid service, you can see the 202.100.0.100 record of client 192.168.3.1 accessing the web server.

# tail/usr/local/squid/var/logs/access log 1582563190.161 9372 192.168.3.1 TCP_MISS / 200-366 GET http://202.100.0.100/ HIER_DIRECT/202.100.0.100 text/ HTML 1582634544.842 2584 192.168.3.1 TCP_CLIENT_REFRESH_MISS/304 218 GET http://202.100.0.100/ -hier_direct / 202.100.0.100-1582634545.836 340 192.168.3.1 TCP_MISS/404 537 GET http://202.100.0.100/favicon.ico - HIER_DIRECT / 202.100.0.100 192.168.3.1 text/HTML 1582635089.249 344 TCP_CLIENT_REFRESH_MISS/304 218 GET http://202.100.0.100/ - HIER_DIRECT/202.100.0.100 -Copy the code

View Web access logs

HTTPD server/HTTPD server/HTTPD server/HTTPD server/HTTPD server/HTTPD server/HTTPD server The record for 192.168.3.1 is, in effect, being accessed by the proxy server on its behalf.

When the client accesses the Web site again, a new record is added to the Squid access log, but the record in the Web access log does not change and is actually provided by the proxy server through the cache. Unless the Web page is adjusted or forcibly refreshed

[23/Feb/2020:21:19:37 +0800] "GET/HTTP/1.1" 200 30 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/ 7.19.7nss / 3.14.0.0zlib /1.2.3 libidn/1.18 libssh2/1.4.2" 202.100.0.100 - - [24/Feb/2020:00:35:09 +0800] "GET/HTTP/1.1" 200 30 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/ 3.14.0.0zlib /1.2.3 libidn/1.18 libssh2/1.4.2" 202.100.0.1 - - [25/Feb/2020:00:53:09 +0800] "GET/HTTP/1.1" 200 30 "-" "Mozilla / 4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident / 4.0; SLCC2; The.net CLR 2.0.50727; The.net CLR 3.5.30729; The.net CLR 3.0.30729; Media Center PC 6.0)" 202.100.0.100 - - [25/Feb/2020:01:21:09 +0800] "GET/HTTP/1.1" 200 30 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2" 202.100.0.1 - - [25/Feb/2020:20:42:22 +0800] "GET/HTTP/1.1" 304 -" -" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident / 4.0; SLCC2; The.net CLR 2.0.50727; The.net CLR 3.5.30729; The.net CLR 3.0.30729; Media Center PC 6.0)" 202.100.0.1 - - [25/Feb/2020:20:42:25 +0800] "GET/Favicon. ico HTTP/1.1" 404 288 "-" Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident / 4.0; SLCC2; The.net CLR 2.0.50727; The.net CLR 3.5.30729; The.net CLR 3.0.30729; Media Center PC 6.0)" 202.100.0.1 - - [25/Feb/2020:20:51:29 +0800] "GET/HTTP/1.1" 304 -" -" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident / 4.0; SLCC2; The.net CLR 2.0.50727; The.net CLR 3.5.30729; The.net CLR 3.0.30729; Media Center PC (6.0)"Copy the code

Transparent Proxy: A redirection policy that depends on default routes and firewalls. This policy applies to LAN host services.

Requirement description:

1) Squid provides proxy service for clients to access the Internet;

2) LAN devices, the IP address, default gateway, do not need to manually specify the proxy server address, port, etc.;

Configure Squid to support transparent proxy

The Squid service does not support transparent proxy by default and needs to be adjusted. With versions later than 2.6, you can support transparent proxy simply by adding transparent to the http_port configuration line.

# vi /etc/squid.conf http_port 192.168.3.100:3128 transparent # service squid reload // Restart squidCopy the code

Set the redirection policy for the firewall

In transparent proxy, the Squid service is built on the Linux gateway host. If the firewall policy is correctly configured, the Squid service can be processed by using the REDIRECT policy of IPTABLES. It mainly realizes the redirection of the local port and sends the outbound data packets of the visited website protocol HTTP (80) and HTTPS (443) to the Squid proxy server (port 3128).

The REDIRECT redirection policy is also a type of packet control, used in the PREROUTING or OUTPUT chain of the NAT table and the chain that is invoked by it, specifying the target port of the mapping in the form of –to-ports port numbers.

# iptables -t nat -A PREROUTING -i eth0 -s 192.168.3.0/24 -p tcp --dport 80 -j REDIRECT --to-ports 3128
# iptables -t nat -A PREROUTING -i eth0 -s 192.168.3.0/24 -p tcp --dport 443 -j REDIRECT --to-ports 3128
# service iptables save
Copy the code

Verify the use of transparent proxies

Verify the effect of transparent proxy, and remove manually specified proxy server Settings from the client. On the Windows operating system, do not select proxy server in the connection Settings of the browser. On the Linux operating system, run the Unset command to clear the HTTP_PROXY and HTTPS_PROXY variables

# unset HTTP_PROXY HTTPS_PROXY
Copy the code

Then the client accesses the target website through the browser and observes the access logs of Squid proxy server and Web server to verify whether transparent proxy plays a role.

ACL access control

Squid can ensure that the resources managed by it are not illegally used or accessed. At the same time, Squid can also restrict the website specified by the cache according to the specific time interval, and filter the source address, target address, URL path, access time and other conditions.

Squids are used to configure access control. First, SQUids are used to name network resources or network objects, and ACL configuration items are used to define the conditions to be controlled. Second: http_access (which controls access to acL-named objects, allowing or denying access);

Define an ACCESS control list

Each ACL configuration line can define an ACL. The syntax format is as follows:

1ACL List Name List Type List Content ···Copy the code

List name (name) : is the name of the object, which can be specified to identify the control conditions;

List type: it is the type of the network object. It can be IP address, domain name, user name, network port number, protocol, request method, regular expression, etc. It must use the predefined value of Squid and correspond to different types of control conditions.

List content (value) : refers to the value of a certain type of network object. Different types of lists have different corresponding contents. There can be multiple values (separated by Spaces and in the relationship of or).

Common types of acl

type meaning
src The source IP address can be a single IP address, an ADDRESS range, or a subnet
dst The destination IP address can be a single IP address, an IP address range, or a subnet address
myip IP address of the local network interface
srcdomain For the domain to which the customer belongs, the Squid queries the DNS according to the IP address of the customer
dstdomain The domain to which the server belongs matches the URL requested by the client
time Represents a period of time
port A network port that points to another computer
myport Points to the SQUID server’s own network port
proto Protocol used by the client request
method HTTP request method
proxy_auth User name authenticated by SQUID itself
url_regex Regular expressions about urls

When defining and using ACL objects, note the following:

1. The value of an ACL type can be an ACL object of the same type.

2. The names of different types of objects must not be the same.

3. The VALUE of an ACL object can be multiple. If any value is matched, the entire ACL object is considered to be matched.

4. When a review of the same type uses its name repeatedly, squids combine all the values into an object of that name;

5. If the object value is a file name, the contents contained in the file are used as the object value, and the file name must be enclosed in double quotation marks.

When defining an ACL, you need to analyze users’ access requirements and define the control conditions for using proxy services based on the current network environment. Different client addresses, target sites to restrict access to, specific time periods, and so on.

# vi/etc/squid. Conf acl localhost/SRC 192.168.1.0 255.255.255.0 / / source address 192.168.1.0 acl MYLAN SRC 192.168.1.0/24 192.168.3.0/24 // Client network segment ACL to_localhost DST 127.0.0.0/8 // Target ADDRESS 127.0.0.0/8 Network segment ACL MC20 maxconn 20 // Maximum concurrent connection 20 ACL WORKTIME time MTWHF 9:00-18:00 // Time: Monday to Friday 9:00-18:00Copy the code

In addition to the preceding parameters, you can also configure the acl based on the common ACL types in the preceding table.

If there are a large number of objects of the same class, you can use an independent folder to save the objects, specify the corresponding file location in the ACL configuration list, or create a blacklist file for the target address.

# mkdir /etc/squid # CD /etc/squid # vi heimingdanip.list // Blacklist target IP address list # vi mubiaoyuip.list // domain target IP address list # vi The/etc/squid. Conf acl HEIMINGDANIP DST "/ etc/squid/HEIMINGDANIP. List" / / call the specified file list acl MUBIAOYUIP dstdomain "/ etc/squid/mubiaoyumingip list"Copy the code

Setting access Permissions

The purpose of defining an ACL object is to control access to requests that match the object, not by the ACL option, but by the HTTP_access or ICP_access option. The HTTP_access configuration line must be placed after the corresponding ACL configuration line. Each HTTP_ACCESS configuration line confirms an access control rule.

Http_access formats:

http_access < allow | deny > [!] ACL object 1 [!] ACL object 2Copy the code

Allow: allows deny: denies

ACL object: a network object defined by the ACL option. There can be multiple network objects. Symbols represent non-operations, objects that are the opposite of ACL objects.

In the HTTP_access rule, multiple access control list names can be contained at the same time. The list names are separated by Spaces.

Is the relationship between and, the restriction is implemented only when all the conditions corresponding to the ACL are met.

If an inverse condition is required, you can add! Before the acl. Symbols.

# vi /etc/squid.conf http_access deny MYLAN MEDIAFILE // Http_access deny MYLAN HEIMINGDANIP // Prohibit client access list of IP address http_access deny MYLAN MUBIAOYUIP / / prohibited client access domain in blacklist http_access deny MYLAN MC20 / / client concurrent connection is interrupted when more than 20 Http_access Allow MYLAN WORKTIME // Allow clients to access the Internet during working hours http_access deny all // By default, all clients are forbidden to use the proxyCopy the code

Squid processes the HTTP_access option by matching the client request to the ACL object in the HTTP_access option. When the request matches each ACL object, allow or deny is executed, as long as the request does not match one of the multiple ACL objects. Http_access is invalid and will not perform any of the specified operations.

Multiple HTTP_access options. If a request matches one of the HTTP_access options, the action specified by HTTP_access will be performed. If none of the http_access options match, the action specified by the last HTTP_access option will be performed.

In general, the most common control rules are placed first to reduce the load on squids.

In the access control policy, the access control policy is denied before being allowed or allowed after being denied. The last rule can be set as the default policy, which is http_access Allow ALL or HTTP_access deny all.

In this Squid environment, use the following four access control configuration options to configure the Squid. After the configuration is complete, restart the Squid service for verification.

# vi /etc/squid acl sorce SRC 192.168.3.1 // Define the address acl mubiao DST 202.100.0.0/24 // Define the target address http_access deny of the Intranet client Sorce mubiao // reject Intranet addresses to external addresses (http_access reference list) HTTP_access allow all // define default rule to allow all # service squid reloadCopy the code

If the client’s proxy access request is rejected by the Squid service, an ERROR page will be displayed. The ERROR page will also be reported according to the conditions you restrict.

The nuggets years | 2021 theme activities I grow half of the campaign is under way on the road…

Recommended reading

Iptables firewall (a) | 4 table / 5 chain, packet matching process, write the iptables rules

The iptables firewall (2) | SNAT/DNAT strategy and application

Iptables firewall (3) | rules of export/import, use firewall script

On RHEL 6, DHCP+TFTP+FTP+PXE+Kickstart implements unattended installation

Use Rsync backup tool and configure Rsync +inotify real-time synchronization in Linux environment

Original is not easy, if you think this article is a little useful to you, please give me a like, comment or forward for this article, because this will be my power to output more quality articles, thanks!

By the way, dig friends remember to give me a free attention yo! In case you get lost and you can’t find me next time.

See you next time!