preface

Angular, Vue and React are used for single-page website development. When users browse, the browser dynamically parses JS and presents the final page, providing better user experience and improved site performance.

However, web crawler will not dynamically parse Js, access to all urls will only get the code in the project entry file, can not get specific content, so it can not do website SEO.

Use prerender. IO to render a web page and then return it to a web crawler for indirect web page parsing. Prerender is relatively simple to configure, requires no change to the project source code, and has zero code intrusion, making it a good solution.

The target

Set up Prerender rendering services based on Centos 7 and Nginx to Prerender web pages for Angular projects

Prerendering flow chart

Run the process

Installing middleware

  1. First sign in prerender. IO and get a personal token
    token
  2. According to the development document, configure the corresponding middleware, such as Nginx, Apache, etc.
  3. To configure Nginx middleware, refer to the following configuration:
server {
    listen 80;
    server_name example.com;
 
    root   /path/to/your/root;
    index  index.html;

    location / {
        try_files $uri @prerender;
    }
 
    location @prerender {
        # Replace YOUR_TOKEN with your personal token
        proxy_set_header X-Prerender-Token YOUR_TOKEN;
        
        set $prerender 0;
        if ($http_user_agent~ *"googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
            set $prerender 1;
        }
        if ($args ~ "_escaped_fragment_") {
            set $prerender 1;
        }
        if ($http_user_agent ~ "Prerender") {
            set $prerender 0;
        }
        if ($uri~ *"\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls |mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") {
            set $prerender 0;
        }
        
        #resolve using Google's DNS server to force DNS resolution and prevent caching of IPsResolver 8.8.8.8;if ($prerender = 1) {
            
            IO to replace your own prerender service, such as 127.0.0.1:3000
            set $prerender "service.prerender.io";
            rewrite .* /$scheme: / /$host$request_uri? break;
            proxy_pass http://$prerender;
        }
        if ($prerender = 0) {
            rewrite .* /index.html break; }}}Copy the code

The reference configuration: gist.github.com/thoop/81658…

  1. Check the nginx configuration and restart nginx
nginx -t
service nginx restart
Copy the code
  1. Middleware installation completed

Install the Prerender service

  1. Install the Node environment on the server

  2. Download the Prerender service

git clone https://github.com/prerender/prerender.git
Copy the code

If the Git service is not installed, manually download it from Github, upload it to /usr, and decompress it to the current directory. 3. Install NPM dependencies

cd /usr/prerender
# Phantomjs official download address will time out, here to re-specify its download address as Taobao image
export PHANTOMJS_CDNURL=https://npm.taobao.org/mirrors/phantomjs
npm install
Copy the code

The file structure is as follows:

File structure

  1. Run the server. Js
Start server. js and listen on port 3000 by default
node server.js
Copy the code

At this point, if you don’t have Chrome installed before, it will start up and say failed to start Chrome, you didn’t detect Chrome, you’re ready to install Chrome why install Chrome, because Prerender doesn’t take care of actual web page parsing, Prerender only takes care of pre-parses and pre-parses. Chrome actually takes care of web page parsing.

Install the Chrome

  1. You cannot access Google in China, so you need to configure the yum source and create the Google-chrome. repo file in the /etc/yum.repos.d/ directory
cd /ect/yum.repos.d/
touch google-chrome.repo
Copy the code
  1. Write vi google-chrome.repo
[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
Copy the code
  1. Installation operation
# Domestic recommendation
yum -y install google-chrome-stable --nogpgcheck
Copy the code
  1. Installation Path After the installation is successful, the Chrome installation path is/opt/google/chromeBy default, user root cannot run Chrome directly, so you can create another user such as Other to run chrome
cd /opt/google/chrome
su other
./chrome
Copy the code
  1. Chrome installation completed

Start the prerender. IO service

  1. The other user is running server.js again
su other
cd /usr/prerender
node ./server.js
Copy the code

It should start successfully, and you can see that the service listens for the startup result of port 3000:

Start the result

if ($prerender = 1) {
            
           # change as follows:
            # set $prerender "service.prerender.io";
            set $prerender "127.0.0.1:3000";
            rewrite .* /$scheme: / /$host$request_uri? break;
            proxy_pass http://$prerender;
        }
Copy the code
  1. Save and restart Nginx
  2. Start the Prerender service again
nohup node ./server.js &
Copy the code

The nohup command is used to add the service to the daemon process to prevent the service from being shut down due to SSH window closing. For details, see Setting Jar in Linux

  1. If the firewall is enabled, add port 3000 to the firewall
Firewall - CMD - zone = public - add - port = 3000 / TCP - permanent# restart firewallFirewall - CMD - reloadCopy the code
  1. At this point, the Prerender service is installed and started successfully
  2. Check the port
    Check the port

    Node, Google-Chrome, and Nginx services should all run in the background

test

If you use html5 push state (recommended):
Just add this meta tag to the <head> of your pages

<meta name="fragment" content="!"> 
Copy the code
If your URLs look like this:
http://www.example.com/user/1 

Then access your URLs like this:
http://www.example.com/user/1?_escaped_fragment_=
Copy the code
If you use the hashbang (#!) :
If your URLs look like this:
http://www.example.com/#! /user/1

Then access your URLs like this:
http://www.example.com/?_escaped_fragment_=/user/1
Copy the code
Run the curl command
curl http://www.example.com/user/1?_escaped_fragment_=
Copy the code

Before prerender service is configured, only the contents of index.html are returned. If the prerender service is configured successfully, the parsed contents are returned.