Nginx is a very popular open source Web server serving millions of applications around the world. Nginx is second only to Apache, thanks to its popularity as a Web server (it can also act as a reverse proxy, HTTP cache, and load balancer) in the way it efficiently delivers static content and overall performance.

From an operational and security perspective, Nginx is at a critical point in the application architecture and requires close monitoring at all times. The Elastic Stack (Elasticsearch, Logstash, Kibana and Beats) is the most popular open source log management and log analysis platform in the world and provides engineers with a very simple and effective way to monitor Nginx.

In this article, we will provide steps to set up pipes for Nginx logs and start monitoring them.

 

Nginx logging foundation

Nginx provides users with a variety of logging options, including logging to file, conditional logging, and syslog logging. Nginx generates two types of logs that can be used for operational monitoring and troubleshooting: error logs and access logs. By default, both logs are usually located under /var/log/nginx, but this location can vary from system to system.

liuxg@liuxg:/var/log/nginx$ ls
access.log  error.log
Copy the code

Nginx error logs

Error logs contain diagnostic information that can be used to troubleshoot operational problems. The Nginx error_log directive can be used to specify the path and severity of log files and can be used in the main HTTP, mail, stream, server, location contexts (in that order). For example, we can specify the path to the error log and the level of severity to log using the following command:

error_log /path/to/log debug;
Copy the code

The above configuration is set in the Nginx configuration file. In the case of my Ubuntu Linux, you can find the configuration file:

$ pwd /etc/nginx liuxg@liuxgu:/etc/nginx$ ls conf.d koi-win nginx.conf sites-available win-utf fastcgi_params mime.types  nginx.conf.org sites-enabled koi-utf modules scgi_params uwsgi_paramsCopy the code

In the nginx.conf configuration file above, we can configure the path and severity of this error log:

user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; }...Copy the code

Log example:

2020/02/26 17:20:33 [error] 7722#7722: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.43.192, server: _, request: “The GET/HTTP / 1.1”, upstream: “http://127.0.0.1:3001/”, host: “192.168.43.192”

Nginx access logs

The access log contains information about all requests sent to and serviced by Nginx. As such, they are a valuable resource for performance monitoring as well as security. The default format for Nginx access logs is the composite format, but it may vary between distributions. As with error logging, you can use the access_log directive to set the log file path and log format.

Log example:

::1 – – [26/Feb/2020:16:46:52 +0800] “GET/HTTP/1.1″ Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36”

 

configuration

The easiest way to send Nginx logs to the Elastic Stack is to use Filebeat.

In earlier versions of the Elastic Stack, Logstash played a key role in the Nginx logging pipeline — processing logs and geographic enhancement. With the advent of the Filebeat module, this can be done without a Logstash, making it much easier to set up the Nginx logging pipeline.

In today’s experiment, we will use the following configuration:

In the above configuration, I have a native Elasticsearch and Kibana running on my MacOS and Ubuntu 18.04 Linux running on another VM. In Ubuntu 18.04, we run the following software:

• Nginx: Open source high-performance HTTP and reverse proxy Web server
• Nodejs: Runs a local Web server
• Filebeat: Used to import Nginx logs into Elasticsearch

The installation

To complete our setup, we do the following installation:

Install Elasticseach

If you don’t already have Elasticsearch installed, follow my tutorial on How to Install Elasticsearch on Linux, MacOS, and Windows to install Elasticsearch. Since our Elastic Stack needs to be accessed by another Ubuntu VM, we need to configure our Elasticsearch. Start by using an editor to open the elasticSearch.yml configuration file in the config directory. We need to change the IP address of network.host. On your Mac and Linux machines, we can use:

$ ifconfig
Copy the code

To see the IP address of our machine. In my case, the IP address of my MacOS machine is: 192.168.43.220.

Above we set network.host to “_site_” to indicate that it is bound to the IP address of our local computer. See the network. Host instructions for Elasticsearch for details.

We must also add discovery.type: single-node at the end of elasticSearch. yml to indicate that we are a single node.

After changing our IP address, let’s save elasticSearch.yml. Then re-run our ElasticSearch. We can enter the IP address we just entered in a browser and add port number 9200. Check to see if our ElasticSearch is working properly.

Install Kibana

We can install Kibana as described in “How to Install Kibana in Elastic Stack on Linux, MacOS, and Windows”. Since the IP address of our Elasticsearch has changed, we have to modify our Kibana configuration file. We use our favorite editor to open the kibana.yml file in the config directory and find server.host. Change its value to the IP address of your computer. In my case:

Find elasticSearch. hosts and enter your OWN IP address:

Save our Kibana.yml file and run our Kibana. Enter your IP address and port 5601 in the browser address:

If the configuration is successful, we can see the above screen.

 

Install Nodejs

We install our Nodes on Ubuntu OS with the following command:

sudo apt update
sudo apt install nodejs
Copy the code

If the package in the repository fits your needs, that’s all you need to do to set up Node.js. In most cases, you’ll also need to install the Node.js package manager NPM. You can do this by typing:

sudo apt install npm
Copy the code

This will allow you to install modules and packages to use with Node.js.

We can check our nodejs version with the following command:

nodejs -v
Copy the code

Next, we’ll install and run a Web server with our own NodeJS. Let’s start by downloading my simple Nodejs code:

git clone https://github.com/liu-xiao-guo/samplenodejs
Copy the code

After we download the code, we can run the following command in the root directory of the project:

npm install
Copy the code

Our NodeJS is a simple Web server based on the Express framework. The command above will help us install all the NodeJS modules we need. Next, we use the following command to start our Web server. It runs on port 3000:

npm start
Copy the code

This can be done in our Ubuntu OS browser with the following address: localhost:3000/hello

You can also open this page on your host machine without starting your Firewall:

If you can see the above two outputs, your NodeJS application is running successfully.

Install Nginx

Nginx is available in Ubuntu’s default repository, so installation is very simple.

Since this is our first interaction with the APT packaging system in this session, we will update the local package index so that we can access the latest package list. After that, we can install nginx:

sudo apt-get update
sudo apt-get install nginx
Copy the code

Once nginx has been successfully installed, we can check whether the nginx service has been successfully started by using the following command:

systemctl status nginx
Copy the code

$systemctl status nginx ● nginx.service - A high performance Web server and A reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-02-26 17:21:21 CST; 17h ago Docs: man:nginx(8) Process: 7740 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx Process: 7743 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/S Process: 7742 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code = exited, s Main PID: 7744 (nginx) Tasks: 7 (4915) limit: CGroup: / system. Slice/nginx service ├ ─ 7744 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; 7745 Nginx: School Exercises ─7746 Nginx: School Exercises for Workers Class exercises ─ 772 Nginx: Class Exercises ─ 772 Liuxg Systemd 2月 26 17:47:51 liuxg Systemd [1]: Starting A High Performance Web Server and A Reverse Proxy Server 2月 26 17:21:21 Liuxg Systemd [1] Started A high performance web server and a reverse proxy serverCopy the code

If we want to stop nginx, we can use the following command:

sudo systemctl stop nginx
Copy the code

To start the Web server at stop time, type:

sudo systemctl start nginx
Copy the code

To stop and then start the service again, type:

sudo systemctl restart nginx
Copy the code

Adjust the firewall

Before we can test Nginx, we need to reconfigure the firewall software to allow access to the service. Nginx registers itself as a service of the UFW (our firewall) after installation. This makes it fairly easy to allow Nginx access.

We can list the application configurations that the UFW knows how to use by typing:

sudo ufw app list
Copy the code

You should get a list of application configuration files:

Available applications:
  CUPS
  Nginx Full
  Nginx HTTP
  Nginx HTTPS
  OpenSSH
Copy the code

As you can see, Nginx provides three configuration files:

• Nginx Full: This configuration file opens both port 80 (normal, unencrypted network traffic) and port 443 (TLS/SSL-encrypted traffic)
• Nginx HTTP: This configuration file only opens port 80 (normal, unencrypted network traffic)
• Nginx HTTPS: this configuration file opens only on port 443 (TLS/SSL encrypted traffic)

It is recommended that you enable the most restrictive profile, which will still allow the traffic you configure. Since we have not configured SSL for the server, for this tutorial we only need to allow traffic through on port 80.

You can enable this feature by typing:

sudo ufw allow 'Nginx HTTP'
Copy the code

You can verify the changes by typing:

sudo ufw status
Copy the code

$ sudo ufw status
Status: inactive
Copy the code

If the above command returns inactive, our firewall has not been opened. In this case in our host MacOS machines browser address http://192.168.43.192:3000/ can see the output. If we want our Firewall to work properly, we can type the following command:

sudo ufw enable
Copy the code

$ sudo ufw enable
Firewall is active and enabled on system startup
Copy the code

Because the firewall is already working. This time we again on MacOS browser to open the address http://192.168.43.192:3000/ will not see any content. This is because our nginx only opens the 80 port.

If we want to disable the firewall, we can type the following command to disable the firewall:

sudo ufw disable
Copy the code

$ sudo ufw disable
Firewall stopped and disabled on system startup
Copy the code

Set Nginx as the reverse proxy server

In the above, we have seen, when we turn on the firewall, we can’t be accessed outside of the http://192.168.43.192:3000/ address.

Now that your application is running and listening on localhost, you need to provide users with a way to access it. To do this, we set up the Nginx Web server as a reverse proxy. We set up the nginx configuration in /etc/nginx/sites-available /default. Open the file for editing:

sudo vi /etc/nginx/sites-available/default
Copy the code

Within the server block, you should have an existing location/block. Replace the contents of the block with the following configuration. If your application is set to listen on another port, update the highlighted part with the correct port number.

/etc/nginx/sites-available/default

. . . location / { proxy_pass http://localhost:3000; Proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; }...Copy the code

Above, we put the port address 3000 into the Settings. You can configure it according to the port of your server. Assuming our server is available on example.com, accessing example.com/ through a Web browser sends the request to the localhost…

When we are done with the above configuration, type the following to make sure we have not introduced any syntax errors:

sudo nginx -t
Copy the code

Next, restart Nginx:

sudo systemctl restart nginx
Copy the code

After we configure, we can re-enter the address on our host MacOS:

This time we can see that even with the Firewall running, external requests can access the web server running at port localhost:3000.

 

Install Filebeat

Installing Filebeat on Ubuntu is also pretty straightforward. We can open our Kibana first.

Let’s click the “Add Log Data” button:

 

Above we click “Nginx logs” :

Since Ubuntu is a Debian system, we chose DEB. We follow the instructions above to complete our installation.

To put it simply:

Download Filebeat and install it

The curl - L - O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.5.0-amd64.deb sudo DPKG -i Filebeat 7.5.0 - amd64. DebCopy the code

With value Filebeat

Modify/etc/filebeat/filebeat yml to set the connection information:

output.elasticsearch:
  hosts: ["<es_url>"]
  username: "elastic"
  password: "<password>"
setup.kibana:
  host: "<kibana_url>"
Copy the code

<password> is the password of the Elastic user, < es_URL > is the URL of Elasticsearch, and < Kibana_URL > is the URL of Kibana. In my case:

Start and configure the nginx module

sudo filebeat modules enable nginx
Copy the code

According to your requirements, we can modify/etc/appropriately filebeat/modules. D/nginx. Yml configuration file. We don’t have to make any changes to our practice.

Start the Filebeat

The setup command loads the Kibana dashboard. If a dashboard has been configured, ignore this command.

sudo filebeat setup
Copy the code

The output of the above command is:

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Loaded machine learning job configurations
Loaded Ingest pipelines
Copy the code

We start Filebeat:

sudo service filebeat start
Copy the code

Check Nginx data

 

Our installation is now complete.

 

Analyze Nginx data

We opened Kibana:

We click “[Filebeat Nginx] Overview ECS” :

At this point, we are done monitoring Nginx logs. Above we can see all the information about Nginx.

 

Reference:

【 1 】 www.digitalocean.com/community/t…

(2) www.digitalocean.com/community/t…

【 3 】 www.digitalocean.com/community/t…

[4] www.digitalocean.com/community/t…