preface
I still remember in my last company, one of the big guys built a surveillance system and wanted to study how he did it. Of course, we are not the people who pat their heads on the head and do what others do. Here’s a look at the benefits of such a platform.
background
Front end system first of all, why should we do for you, look at this table below, can clearly see that the front end performance increase the value of the product, or pretty helpful, but if we can real-time collected these information, and to monitor implementation, make the whole product in the product line has been to maintain efficient operation, this is our purpose.
| performance | earnings |
|---|---|
| Google’s delay400ms | Searches are down0.59% |
| Bing delay2s | Drop in revenue4.3% |
| Yahoo delay400ms | Flow down5-9% |
| Mozilla pages open less2.2 s | Download increase15.4% |
| Netflix open Gzip | Improved performance by 13.25% Reduced bandwidth50% |
Secondly, it is also good for the products we release, so that we can find our mistakes in time. If a product is in a new iteration, an unspeakable error occurs.
Right! It’s just indescribable. We can’t wait for users to complain. By then, the day lilies are cold.
start
Based on the above, we began to build a front-end monitoring and simple control platform. (Although there are many such systems on the market, such as ELK, I can’t help myself.)
It has to be easy.
Guys, forgive me, this is as far as I can get you.
Here we go.
These are some of the things we need to do.
To collect information
To make a surveillance system, first we have to have an object. The object of our surveillance! Object! Object! Object.
I wrote a page in my system that looked like this,
<body>
<div>2</div>
<div>2</div>
<div>2</div>
<div>2</div>
<div>2</div>
<div>2</div>
</body>
Copy the codeYes, this is the page we’re monitoring. This… I’m not lazy.
And then I designed three pieces of data
- Page load time
- Collect statistics about devices used by users
- Error statistics
Page load time
window.logInfo = {}; // Count page load time
window.logInfo.openTime = performance.timing.navigationStart;
window.logInfo.whiteScreenTime = +new Date() - window.logInfo.openTime;
document.addEventListener('DOMContentLoaded'.function (event) {
window.logInfo.readyTime = +new Date() - window.logInfo.openTime;
});
window.onload = function () {
window.logInfo.allloadTime = +new Date() - window.logInfo.openTime;
window.logInfo.nowTime = new Date().getTime();
var timname = {
whiteScreenTime: 'White screen time'.readyTime: 'User available time'.allloadTime: 'Total Download time'.mobile: 'Use equipment'.nowTime: 'time'};var logStr = ' ';
for (var i in timname) {
console.warn(timname[i] + ':' + window.logInfo[i] + 'ms');
if (i === 'mobile') {
logStr += '&' + i + '=' + window.logInfo[i];
} else {
logStr += '&' + i + '=' + window.logInfo[i]; }} (new Image()).src = '/action? ' + logStr;
};
Copy the codeCollect statistics about devices used by users
window.logInfo.mobile = mobileType();
function mobileType() {
var u = navigator.userAgent, app = navigator.appVersion;
var type = {// Mobile terminal browser version information
ios:!!!!! u.match(/\(i[^;] +; ( U;) ? CPU.+Mac OS X/), / / ios terminal
iPad: u.indexOf('iPad') > -1./ / whether the device
android: u.indexOf('Android') > -1 || u.indexOf('Linux') > -1.// An Android terminal or uc browser
iPhone: u.indexOf('iPhone') > -1 || u.indexOf('Mac') > -1.// Whether the browser is iPhone or QQHD
trident: u.indexOf('Trident') > -1./ / IE kernel
presto: u.indexOf('Presto') > -1./ / opera kernel
webKit: u.indexOf('AppleWebKit') > -1.// Apple, Google kernel
gecko: u.indexOf('Gecko') > -1 && u.indexOf('KHTML') = = -1.// The Firefox kernel
mobile:!!!!! u.match(/AppleWebKit.*Mobile/i) | |!!!!! u.match(/MIDP|SymbianOS|NOKIA|SAMSUNG|LG|NEC|TCL|Alcatel|BIRD|DBTEL|Dopod|PHILIPS|HAIER|LENOVO|MOT-|Nokia|SonyEricsson|SIE-|Amoi |ZTE/), // Whether it is a mobile terminal
webApp: u.indexOf('Safari') = = -1 // Whether the web should be a program, without a header and a bottom
};
var lists = Object.keys(type);
for(var i = 0; i < lists.length; i++) {
if(type[lists[i]]) {
returnlists[i]; }}}Copy the codeError statistics
window.onload = function () {
window.logInfo.allloadTime = +new Date() - window.logInfo.openTime;
window.logInfo.nowTime = new Date().getTime();
var timname = {
whiteScreenTime: 'White screen time'.readyTime: 'User available time'.allloadTime: 'Total Download time'.mobile: 'Use equipment'.nowTime: 'time'};var logStr = ' ';
for (var i in timname) {
console.warn(timname[i] + ':' + window.logInfo[i] + 'ms');
if (i === 'mobile') {
logStr += '&' + i + '=' + window.logInfo[i];
} else {
logStr += '&' + i + '=' + window.logInfo[i]; }} (new Image()).src = '/action? ' + logStr;
};
var defaults = {
msg:' '.// Error specific information
url:' '.// The url where the error occurred
line:' '.// The error line
col:' '.// The column where the error occurred
nowTime: ' './ / time
};
window.onerror = function(msg,url,line,col,error) {
col = col || (window.event && window.event.errorCharacter) || 0;
defaults.url = url;
defaults.line = line;
defaults.col = col;
defaults.nowTime = new Date().getTime();
if (error && error.stack){
// If the browser has stack information, use it directly
defaults.msg = error.stack.toString();
}else if (arguments.callee){
// Try to fetch stack information through callee
var ext = [];
var fn = arguments.callee.caller;
var floor = 3;
while (fn && (--floor>0)) {
ext.push(fn.toString());
if (fn === fn.caller) {
break;
}
fn = fn.caller;
}
ext = ext.join(",");
defaults.msg = error.stack.toString();
}
var str = ' '
for(var i in defaults) {
// console.log(i,defaults[i]);
if(defaults[i] === null || defaults[i] === undefined) {
defaults[i] = 'null';
}
str += '&'+ i + '=' + defaults[i].toString();
}
srt = str.replace('&'.' ').replace('\n'.' ').replace(/\s/g.' ');
(new Image()).src = '/error? ' + srt;
}
Copy the codeThat’s all you need to do to collect data, either by sending /action requests or by sending /error requests, which can be customized, but I’m just talking about how the process works.
Then I processed and recorded all the requests through one of my background express.js, and the data after recording is like this.
User_ip = 127.0.0.1 & whiteScreenTime = 185 & readyTime = 192 & allloadTime mobile = webKit&nowTime = = 208 & 1513071388941Copy the codeThe data processing
Here I wrote a script to parse, parse.js, here is not specific, look at the source code. Let me show you the parsed data.
I store the data in CVS format, and I also support json format export for the following diagrams, but you need to configure the visual interface later.
Here’s the data.
charts/csvData/2017-12-16time.csv
Time, bad time, the user can operation time, total download time, 1513427051482137137153, 1513427065080470471507, 1513427080040127127143 1513428714345274275323 1513428733583267268317 1513428743167268268317 1513428754796276276328Copy the codeThe data show
I’m using highcharts.js here
I will not explain the specific configuration, you can go to the official website to check.
Below is a visual chart showing the information for each time of day.
The interface may not be particularly beautiful, please forgive me.
The environment
The node > = 6.0.0
Redis > = server
Here I would like to explain that if this online environment is deployed, if every record is recorded, it will consume a lot of memory, so I set up a layer of Redis, in order to prevent the impact of heavy traffic, and then can be stored at intervals.
const express = require('express');
const performance = require('./lib/performance.js');
const app = express();
const router = express.Router();
router.get('/'.function (req, res, next) {
req.url = './index.html';
next();
});
app.use(router);
app.use(performance({
time: 10.// The unit is seconds
originalDir: './originalData'.// Directory of data
errorDir: './errorData' // The directory where the error was reported
}))
app.use(express.static('/'));
const server = app.listen(3000)
Copy the codeYou can set the default time here, but I’m going to do it in 10 seconds for demo purposes. I usually do it once a minute.
Code address: github.com/hua1995116/…
If you have good suggestions and optimization plan, please also put forward to me in Issues.
Advanced (an actual combat chestnut using monitoring platform)
I used this platform to monitor one of my projects. If you just play, please just read the original system address above, you can ignore my paragraph, after all, my system is not perfect.
Online demo: www.qiufengh.com/#/
Monitor demo:qiufengh.com :8080/
Project: github.com/hua1995116/…
Here I have set logging every 1 minute.
// Monitor import
app.use(performance({
time: 60.// The unit is seconds
originalDir: './originalData'.// Directory of data
errorDir: './errorData' // The directory where the error was reported
}))
Copy the codeAnd parsing every 10 minutes.
function setPrase() {
setInterval(function(){
parseData();
}, 1000 * 60 * 10);
}
Copy the code
2017-12-20
This article only provides an idea, if you want professional, you can use professional statistical analysis tools such as ELK.