This is the 17th day of my participation in Gwen Challenge

A lifelong learner, practitioner and sharer committed to the path of technology, a busy and occasionally lazy original blogger, an occasionally boring and occasionally humorous teenager.

Welcome to dig friends wechat search “Jie Ge’s IT journey” attention!

A Squid log analysis tool

preface

In the last article, we introduced the basic concept of Squid proxy server, the working mechanism of Squid proxy, the basic type of proxy, installing Squid service, building traditional proxy, transparent proxy and ACL access control. Squid proxy server | 2021 Summary “, so today, introduce you a commonly used Squid log analysis software, hope to help students in the future daily work can use it.

Sarg: Full name: Squid Analysis Report Generator, Squid Analysis Report Generator is a Squid log Analysis tool. It uses HTML format to list the Internet website information, time occupation information, ranking, connection times, visits and other related information of users.

Squid log analysis software: www.squid-cache.org/Misc/log-an…

I. Sarg deployment process

Install the GD library

# yum -y install gd gd-devel
Copy the code

Install the sarg

/usr/local/sarg # CD /usr/local/sarg/ # tar ZXF sarg-2.3.7.tar.gz # CD sarg-2.3.7 #./configure --prefix=/usr/local/sarg/ -sysconfdir=/etc/sarg --enable-extraprotection && make && make installCopy the code

Meanings of configuration items:

  • -sysconfdir=/etc/sarg: configuration file directory
  • Enable-extraprotection: Adds extra security protection

configuration

# vi/etc/sarg/sarg. Conf 7 access_log/usr/local/squid/var/logs/access log/squid/specify access log file title 25 "squids User Access Reports" // web page Title 120 output_dir /var/www/ HTML /sarg // Output directory of sarg report 178 user_ip no // Display with user name 184 Topuser_sort_field connect Reverse // In the top sort, specify the number of connections, the number of bytes accessed, in descending order 190 user_sort_field connect Reverse // Number of connections in descending order 206 exclude_hosts /usr/local/sarg/noreport // Specifies the list of sites not counted in sorting 257 overwrite_report no // When the date report already exists, 434 charset UTF-8 // Use character set 518 weekdays 0-6 // Specify the weekly cycle of the top sort, 0 is Sunday 525 hours 9-12,14-16,18-20 // Specifies the time period for top sorting. 633 www_document_root /var/wwww/html // Specifies the web page root directoryCopy the code

run

In the preceding configuration, the sites that are not included in the sorting must be saved in the /usr/local/sarg/noreport file. The added domain names are not displayed in the sorting. Execute sarG directly to start a record. It is recommended to set a symbolic link and then execute SARG. Output information will be displayed.

# touch/usr/local/sarg/noreport # ln -s/usr/local/sarg/bin/sarg/usr/local/bin / # sarg sarg: record in the file: 0, reading: 100.00% SARG: No record found SARG: endCopy the code

Sarg command help

  • -a: Specifies the host name or address that is not included in the sorting
  • -b: outputs user agent log files
  • – c: not included in the list of sites sorted specified file called/usr/local/sarg/norecords, for the client to access the site, will not be in the top ranking;
  • -d: specifies the date range
  • -e: specifies the report recipient email
  • -f: Specifies the configuration file
  • -g: specifies the entered date format
  • -h: Help information
  • -i: specifies the user name or IP address for sorting clients
  • -l: Specifies the absolute path of the SQUID log file.
  • -o: specifies the output path of the web report file. You are advised to run sarg as the Webmaster or a non-admin user.
  • -p: uses the IP address as the userID domain
  • -w: specifies the temporary file directory. Ensure that the partition where the temporary file directory resides is large enough and at least 1 GB.

Plan tasks

Sarg can be scheduled and executed on a regular basis.

/usr/local/sarg/daily.sh // daily.sh /bin/bash #Get current date TODAY=$(date +%d/%m/%Y) #Get one week ago today YESTERDAY=$(date --date "1 day ago" +%d/%m/%Y) /usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -o /var/www/html/sarg -z -d $YESTERDAY-$TODAY &> /dev/null exit 0 # chmod +x /usr/local/sarg/daily.sh # crontab -e 00 00 * * * /usr/local/sarg/daily.sh # chkconfig crond onCopy the code

In addition to daily reports, you can also write weekly reports, monthly reports and send reports by email. Please refer to the following scripts:

Weekly Report: ------------------------------------- #! /bin/bash #Get current date TODAY=$(date +%d/%m/%Y) #Get one week ago today YESTERDAY=$(date --date "1 week ago" +%d/%m/%Y) /usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -o /usr/local/apache/htdocs/reports/weekly -z -d $$TODAY YESTERDAY - exit 0 month report: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- #! /bin/bash #Get current date TODAY=$(date +%d/%m/%Y) #Get one week ago today YESTERDAY=$(date --date "1 month ago" +%d/%m/%Y) /usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -o /usr/local/apache/htdocs/reports/monthly -z -d $YESTERDAY - $TODAY squid/usr/local/squid/bin / -k rotate exit 0 through email report: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- #! /bin/bash #Get current date TODAY=$(date +%d/%m/%Y) #Get one week ago today YESTERDAY=$(date --date "1 day ago" +%d/%m/%Y) /usr/local/bin/sqmgrlog -l /usr/local/squid/logs/access.log -e [email protected] -z -d $YESTERDAY-$TODAY exit 0Copy the code

4. Common problems

Symptom The RPM package cannot be installed and an error occurs

warning: *.rpm: Header V3 RSA/SHA256 Signature, keykey ID c105b9de:
Copy the code

Solutions:

RPM –force –nodeps RPM -ivh *. RPM –force –nodeps RPM -ivh *. RPM –force –nodeps

Nodeps means to ignore dependencies. In Linux, dependencies exist between different software. With these two options, you can ignore these dependencies and force installation or uninstallation.

Such as:

RPM -ivh gd-devel-2.0.35-11.el6.x86_64. RPM --force --nodepsCopy the code

Or try uninstalling:

With man RPM, Find — AllMatches solves this problem.

Such as:

# rpm -e --allmatches --nodeps gd*
Copy the code

An error occurred when executing the sarg command. Procedure

# sarg
SARG: Unknown sort order "BYTES" for parameter "topuser_sort_field"
Copy the code

Solutions:

Edit the sarg.conf configuration file and remove BYTES from the topuser_sort_field connect BYTES reverse line 184;

# vi /etc/sarg/sarg.conf
184 topuser_sort_field connect BYTES reverse
Copy the code

Then execute the command sarg;

# sarg sarg: record in file: 0, reading: 100.00% Sarg: no record found sarg: endCopy the code

Recommended reading

A long article on the most commonly used open source Squid proxy servers | 2021 summary

Original is not easy, if you think this article is useful to you, please kindly like, comment or forward this article, because this will be my power to output more high-quality articles, thank you!

By the way, please give me some free attention! In case you get lost and don’t find me next time.

See you next time!