An overview of the
Logging is a very common development habit for backend development, and usually we use try… Catch code block is used to actively catch errors. For each interface call, time consumption of each interface call is also recorded, so that we can monitor server interface performance and troubleshoot problems.
When I first entered the company, WHEN I was developing node.js interface, I was not used to having to log on the server through the jumper every time I checked problems. Later, I gradually got used to this way.
Here’s an example:
@parma req, res */
exports.getList = async function (req, res) {
// Get the request parameters
const openId = req.session.userinfo.openId;
logger.info(`handler getList, user openId is ${openId}`);
try {
// Get the list data
const startTime = new Date().getTime();
let res = await ListService.getListFromDB(openId);
logger.info(`handler getList, ListService.getListFromDB cost time The ${new Date().getTime() - startDate}`);
// Process the data and return it to the front end
// ...
} catch(error) {
logger.error(`handler getList is error, The ${JSON.stringify(error)}`); }};Copy the codeThe following code is often used in node.js interfaces to count the time spent querying DB, or RPC service calls, in order to detect performance bottlenecks and optimize performance; Or try for exceptions… Catch takes the initiative to catch, so that the problem can be traced back at any time, the scene of the problem can be restored, and the bug can be fixed.
What about the front end? Take a look at the following scenario.
Recently, while working on a requirements development project, we came across an occasional webGL rendering image failure, or image parsing failure, where we might not know which image was being parsed or rendered. Or for another requirement recently developed, we will make a requirement about webGL rendering time optimization and image preloading. If there is no performance monitoring, how can we count the ratio of rendering optimization and image preloading optimization, and how can we prove the value of what we do? It may be through the black box test of the test students to record the screen before and after optimization, and analyze how many frames of images have passed from entering the page to the completion of image rendering. Such data may be inaccurate and one-sided. It is assumed that the test students are not real users, nor can they restore the network environment of real users. Looking back, we found that although logging and performance statistics were done at the server level, anomaly monitoring and performance statistics were performed at the front end of our project. It is necessary to explore the feasibility of front-end performance and exception reporting.
Exception handling
For the front end, we need to catch exceptions in one of two ways:
- Interface call;
- The page logic is incorrect. For example, a blank screen is displayed after a user enters the page.
For interface invocation, client parameters, such as user OS and browser versions and request parameters (such as page ID), need to be reported in the front end. For the problem of whether the page logic is wrong, in addition to the user OS and browser version, usually requires the stack information of the error and the specific error location.
Exception catching method
Global capture
This can be caught with a global listener exception, either window.onerror or addEventListener, as shown in the following example:
window.onerror = function(errorMessage, scriptURI, lineNo, columnNo, error) {
console.log('errorMessage: ' + errorMessage); // Exception information
console.log('scriptURI: ' + scriptURI); // Abnormal file path
console.log('lineNo: ' + lineNo); // Exception line number
console.log('columnNo: ' + columnNo); // Exception column number
console.log('error: ' + error); // Abnormal stack information
// ...
// An exception is reported
};
throw new Error('This is a mistake');
Copy the code
The window.onerror event can be used to obtain specific exception information, URL of exception file, row and column number of exception, and stack information of exception. After the exception is captured, it will be reported to our log server.
Alternatively, the window.addEventListener method is used to report exceptions.
window.addEventListener('error'.function() {
console.log(error);
// ...
// An exception is reported
});
throw new Error('This is a mistake');
Copy the code
try… catch
Using a try… While a catch does a good job of catching exceptions and not causing the page to hang due to an error, try… The catch method is too bloated, and most of the code uses try… Catch wrap, affecting code readability.
Q&A
Cross-domain scripts cannot accurately catch exceptions
Typically, static resources such as JavaScript scripts are placed on a dedicated static resource server, or CDN, as shown in the following example:
<html>
<head>
<title></title>
</head>
<body>
<script type="text/javascript">
/ / in the index. HTML
window.onerror = function(errorMessage, scriptURI, lineNo, columnNo, error) {
console.log('errorMessage: ' + errorMessage); // Exception information
console.log('scriptURI: ' + scriptURI); // Abnormal file path
console.log('lineNo: ' + lineNo); // Exception line number
console.log('columnNo: ' + columnNo); // Exception column number
console.log('error: ' + error); // Abnormal stack information
// ...
// An exception is reported
};
</script>
<script src="./error.js"></script>
</body>
</html>
Copy the code// error.js
throw new Error('This is a mistake');
Copy the code
It turns out that window. onError does not catch the correct exception at all, but returns a Script error.
Solution: Add a crossorigin= “anonymous” to the script tag and add access-Control-allow-Origin to the server.
<script src="http://cdn.xxx.com/index.js" crossorigin="anonymous"></script>
Copy the codesourceMap
Usually code in production is webpack packed and compressed obliquely, so we might run into problems like this, as shown in the figure below:
We find that all the error lines are in the first line. Why? This is because in production, our code is compressed into a single line:
!function(e){var n={};function r(o){if(n[o])return n[o].exports;var t=n[o]={i:o,l:!1.exports: {}};return e[o].call(t.exports,t,t.exports,r),t.l=!0,t.exports}r.m=e,r.c=n,r.d=function(e,n,o){r.o(e,n)||Object.defineProperty(e,n,{enumerable:!0.get:o})},r.r=function(e){"undefined"! =typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule", {value:!0})},r.t=function(e,n){if(1&n&&(e=r(e)),8&n)return e;if(4&n&&"object"= =typeof e&&e&&e.__esModule)return e;var o=Object.create(null);if(r.r(o),Object.defineProperty(o,"default", {enumerable:!0.value:e}),2&n&&"string"! =typeof e)for(var t in e)r.d(o,t,function(n){return e[n]}.bind(null,t));return o},r.n=function(e){var n=e&&e.__esModule?function(){return e.default}:function(){return e};return r.d(n,"a",n),n},r.o=function(e,n){return Object.prototype.hasOwnProperty.call(e,n)},r.p="",r(r.s=0)} ([function(e,n){throw window.onerror=function(e,n,r,o,t){console.log("errorMessage: "+e),console.log("scriptURI: "+n),console.log("lineNo: "+r),console.log("columnNo: "+o),console.log("error: "+t);var l={errorMessage:e||null.scriptURI:n||null.lineNo:r||null.columnNo:o||null.stack:t&&t.stack? t.stack:null};if(XMLHttpRequest){var u=newXMLHttpRequest; u.open("post"."/middleware/errorMsg",!0),u.setRequestHeader("Content-Type"."application/json"),u.send(JSON.stringify(l))}},new Error("This is a mistake.")}]);
Copy the codeIn my development process, I also encountered this problem. When I was developing a functional component library, I used NPM link to use my component library. However, because the component library was packaged by NPM link in the production environment, all the errors were located in the first line.
The solution is to enable Webpack’s source-map, which is a script file generated by webpack that allows the browser to track error locations. Refer to webpack Document here.
Js with devtool: ‘source-map’, as shown below, for example webpack.config.js:
var path = require('path');
module.exports = {
devtool: 'source-map'.mode: 'development'.entry: './client/index.js'.output: {
filename: 'bundle.js'.path: path.resolve(__dirname, 'client')}}Copy the codeGenerate the corresponding source-map after webpack is packed so that the browser can locate the specific error location:
The drawback of enabling Source-Map is compatibility. Source-map is currently supported only by Chrome and Firefox. But we have a solution for that, too. Source-map can be supported using the imported NPM library, see Mozilla /source-map. The NPM library can run on both the client and server, but it is recommended to use source-map parsing when the server uses Node.js to access the received log information to avoid the risk of source code leakage, as shown in the following code:
const express = require('express');
const fs = require('fs');
const router = express.Router();
const sourceMap = require('source-map');
const path = require('path');
const resolve = file= > path.resolve(__dirname, file);
// Define the POST interface
router.get('/error/'.async function(req, res) {
// Get the error object from the front end
let error = JSON.parse(req.query.error);
let url = error.scriptURI; // Zip file path
if (url) {
let fileUrl = url.slice(url.indexOf('client/')) + '.map'; // Map file path
/ / parsing sourceMap
let consumer = await new sourceMap.SourceMapConsumer(fs.readFileSync(resolve('.. / ' + fileUrl), 'utf8')); // Return a Promise object
// Parse the original error data
let result = consumer.originalPositionFor({
line: error.lineNo, // The compressed line number
column: error.columnNo // Compressed column number
});
console.log(result); }});module.exports = router;
Copy the codeAs shown in the figure below, we can see that the specific error line number and column number have been successfully resolved on the server side, which can be recorded in the way of logs to achieve the purpose of front-end anomaly monitoring.
Vue catches exceptions
In my project, I encountered such a problem. I used plug-ins like JS-Tracker to uniformly capture global exceptions and report logs. However, it was found that we could not capture the exceptions of Vue components at all. Onerror will not pass to window.onerror. So how do we uniformly catch exceptions in Vue components?
With a Vue global configuration such as vue.config. errorHandler, you can use handlers that do not catch errors during rendering and viewing of Vue specified components. When this handler is called, it gets an error message and a Vue instance.
Vue.config.errorHandler = function (err, vm, info) {
// handle error
// 'info' is Vue specific error information, such as the lifecycle hook where the error occurred
// Only available in 2.2.0+
}
Copy the codeIn React, the ErrorBoundary component including business components can be used to catch exceptions. With React 16.0+ the new componentDidCatch API, unified exception catching and log reporting can be achieved.
class ErrorBoundary extends React.Component {
constructor(props) {
super(props);
this.state = { hasError: false };
}
componentDidCatch(error, info) {
// Display fallback UI
this.setState({ hasError: true });
// You can also log the error to an error reporting service
logErrorToMyService(error, info);
}
render() {
if (this.state.hasError) {
// You can render any custom fallback UI
return <h1>Something went wrong.</h1>;
}
return this.props.children; }}Copy the codeThe usage is as follows:
<ErrorBoundary>
<MyWidget />
</ErrorBoundary>
Copy the codePerformance monitoring
Simplest performance monitoring
The most common performance monitoring requirement is to count the time between the user’s request for the page and the completion of the rendering of all DOM elements, also known as the first screen load time. DOM provides this interface. Monitor the DOMContentLoaded event of document and the load event of window to count the loading time of the first screen of the page, i.e., all DOM rendering times:
<html>
<head>
<title></title>
<script type="text/javascript">
// Record the start time of page loading
var timerStart = Date.now();
</script>
<! Load static resources such as style resources -->
</head>
<body>
<! Load static JS resource -->
<script type="text/javascript">
document.addEventListener('DOMContentLoaded'.function() {
console.log(DOM mount time:.Date.now() - timerStart);
// Performance logs are reported
});
window.addEventListener('load'.function() {
console.log("Time to complete loading of all resources:".Date.now()-timerStart);
// Performance logs are reported
});
</script>
</body>
</html>
Copy the codeFor frameworks such as Vue or React, where components are rendered asynchronously and then mounted to the DOM, there are not many DOM nodes at page initialization. See the following solution for first screen time capture automation to count rendering times.
performance
However, the monitoring of the above time is too rough. For example, if we want to count the network loading time, DOM parsing time and DOM rendering time of the document, it is not easy to do so. Fortunately, the browser provides the interface window.performance, and the SPECIFIC MDN document can be seen
Almost all browsers support the Window.performance interface. Here’s what you get by printing Window.performance on the console:
As you can see, the window, the performance is mainly consists of the memory, navigation, timing and timeOrigin and onresourcetimingbufferfull method.
navigationThe object provides information about the operation that took place during a specified period of time, including whether the page was loaded or refreshed, how many redirects occurred, and so on.timingObject contains delay-related performance information. This is the main information reported in our page load performance optimization requirements.memoryforChromeA nonstandard extension added to this property that provides an object that can retrieve basic memory usage. This should be considered in other browsersAPICompatible processing.timeOriginReturns a high-precision timestamp of the time when the performance measurement began. As you can see, it’s accurate to four decimal places.onresourcetimingbufferfullMethod, it is a inresourcetimingbufferfullCalled when the event is triggeredevent handler. This event is triggered when the browser’s resource time performance buffer is full. You can anticipate the page by listening for this event triggercrash, Statistics pagecrashProbability for later performance optimization, as shown in the following example:
function buffer_full(event) {
console.log("WARNING: Resource Timing Buffer is FULL!");
performance.setResourceTimingBufferSize(200);
}
function init() {
// Set a callback if the resource buffer becomes filled
performance.onresourcetimingbufferfull = buffer_full;
}
<body onload="init()">
Copy the codeCalculate site performance
Use the performance of timing properties, can get page performance related data, there are a lot of articles have mentioned about using Windows. Performance. The timing records page performance, For example, Alloyteam wrote a study on performance – monitoring web page and program performance. For the meaning of timing attributes, you can use the figure below to understand. The following code is extracted from this article as a tool function reference to calculate website performance:
// Get performance data
var performance = {
// Memory is a nonstandard property, available only in Chrome
// Wealth question: How much memory do I have
memory: {
usedJSHeapSize: 16100000.// JS objects (including V8 internal objects) must take up less memory than totalJSHeapSize
totalJSHeapSize: 35100000.// Available memory
jsHeapSizeLimit: 793000000 // Memory size limit
},
// Philosophical question: Where do I come from?
navigation: {
redirectCount: 0.// The page is redirected several times, if any
type: 0 // 0 is TYPE_NAVIGATENEXT's normal page (not refreshed, not redirected, etc.)
// 1 is the page that TYPE_RELOAD refreshes via window.location.reload()
// 2 i.e. TYPE_BACK_FORWARD page entered through the browser's forward and back buttons (history)
// 255: TYPE_UNDEFINED The page that is not entered in the above way
},
timing: {
// In the same browser context, the timestamp for the previous page (not necessarily the same domain as the current page) to unload, or equal to the fetchStart value if there was no previous page unload
navigationStart: 1441112691935.// Time stamp of previous page unload (same domain as current page), 0 if there is no previous page unload or if previous page is in a different domain than current page
unloadEventStart: 0.// Corresponding to unloadEventStart, returns the timestamp when the callback function bound with the Unload event on the previous page has finished executing
unloadEventEnd: 0.// The time when the first HTTP redirect occurs. The value is 0 only when there is a redirect within the same domain name
redirectStart: 0.// The time when the last HTTP redirect was completed. The value is 0 only when there is a redirect within the same domain name
redirectEnd: 0.// The time when the browser is ready to grab the document using an HTTP request, before checking the local cache
fetchStart: 1441112692155.// Start time of DNS domain name query. If local cache (no DNS query) or persistent connection is used, this value is the same as fetchStart value
domainLookupStart: 1441112692155.// Time when DNS domain name query is completed. If local cache is used (that is, no DNS query is performed) or persistent connection is used, this value is the same as the fetchStart value
domainLookupEnd: 1441112692155.// The time when the HTTP (TCP) connection is started, equal to the fetchStart value if the connection is persistent
// Note that if an error occurs at the transport layer and the connection is re-established, this displays the time when the newly established connection started
connectStart: 1441112692155.// The time HTTP (TCP) takes to complete the connection establishment (complete the handshake), equal to the fetchStart value if the connection is persistent
// Note that if an error occurs at the transport layer and the connection is re-established, this displays the time when the newly established connection is completed
// Note that the handshake is complete, including the establishment of the security connection and the SOCKS authorization
connectEnd: 1441112692155.// The time when the HTTPS connection starts. If the connection is not secure, the value is 0
secureConnectionStart: 0.// The time the HTTP request starts to read the real document (the connection is completed), including reading from the local cache
// When a connection error reconnects, the time of the new connection is displayed here
requestStart: 1441112692158.// The time when HTTP starts receiving the response (the first byte is retrieved), including reading from the local cache
responseStart: 1441112692686.// The time when the HTTP response is fully received (fetched to the last byte), including reading from the local cache
responseEnd: 1441112692687.// Start parsing the rendering time of the DOM tree, document. readyState becomes loading, and the readyStatechange event is thrown
domLoading: 1441112692690.// When the DOM tree is parsed, document. readyState becomes interactive, and readyStatechange events are thrown
// Note that only the DOM tree has been parsed, and no resources within the page have been loaded
domInteractive: 1441112693093.// The time when resources in the web page start loading after DOM parsing is complete
// occurs before the DOMContentLoaded event is thrown
domContentLoadedEventStart: 1441112693093.// The time when the resources in the web page are loaded after the DOM parsing is completed (e.g. the JS script is loaded and executed)
domContentLoadedEventEnd: 1441112693101.// When the DOM tree is parsed and the resource is ready, document. readyState becomes complete and the readyStatechange event is thrown
domComplete: 1441112693214.// The load event is sent to the document, which is when the LOAD callback starts executing
// Note that if no load event is bound, the value is 0
loadEventStart: 1441112693214.// The time when the callback of the load event completes
loadEventEnd: 1441112693215
// alphabetical order
// connectEnd: 1441112692155,
// connectStart: 1441112692155,
// domComplete: 1441112693214,
// domContentLoadedEventEnd: 1441112693101,
// domContentLoadedEventStart: 1441112693093,
// domInteractive: 1441112693093,
// domLoading: 1441112692690,
// domainLookupEnd: 1441112692155,
// domainLookupStart: 1441112692155,
// fetchStart: 1441112692155,
// loadEventEnd: 1441112693215,
// loadEventStart: 1441112693214,
// navigationStart: 1441112691935,
// redirectEnd: 0,
// redirectStart: 0,
// requestStart: 1441112692158,
// responseEnd: 1441112692687,
// responseStart: 1441112692686,
// secureConnectionStart: 0,
// unloadEventEnd: 0,
// unloadEventStart: 0}};Copy the code// Calculate the load time
function getPerformanceTiming() {
var performance = window.performance;
if(! performance) {// Not supported by current browser
console.log('Your browser does not support the Performance interface');
return;
}
var t = performance.timing;
var times = {};
// [Important] The time when the page is loaded
// [reason] This almost represents the amount of time the user has been waiting for the page to be available
times.loadPage = t.loadEventEnd - t.navigationStart;
// [Important] Time to parse DOM tree structure
// Do you have too many nested DOM trees?
times.domReady = t.domComplete - t.responseEnd;
[Important] Redirection time
// [cause] Reject redirection! For example, http://example.com/ should not be written as http://example.com
times.redirect = t.redirectEnd - t.redirectStart;
// Major DNS query time
// [cause] is DNS preloading done? Is there too many different domain names on the page and the domain name query takes too long?
/ / can use HTML 5 Prefetch query DNS, see: [HTML 5 Prefetch] (http://segmentfault.com/a/1190000000633364)
times.lookupDomain = t.domainLookupEnd - t.domainLookupStart;
// Important The time to read the first byte of the page
//【 Reason 】 This can be understood as the time for users to get your resources, did you add the remote computer room, did you add the CDN processing? Did you increase the bandwidth? Did you increase the CPU speed?
// TTFB stands for Time To First Byte
/ / wikipedia: https://en.wikipedia.org/wiki/Time_To_First_Byte
times.ttfb = t.responseStart - t.navigationStart;
// [Important] The time when the content is finished loading
//【 Cause 】 Is the page content gzip compressed, static resources CSS/JS compressed?
times.request = t.responseEnd - t.requestStart;
[Major] The time when the onload callback function is executed
// [cause] Are too many unnecessary operations being performed in the onload callback function? Have you considered lazy loading, load on demand?
times.loadEvent = t.loadEventEnd - t.loadEventStart;
// DNS cache time
times.appcache = t.domainLookupStart - t.fetchStart;
// The time to uninstall the page
times.unloadEvent = t.unloadEventEnd - t.unloadEventStart;
// The time when TCP establishes the connection and completes the handshake
times.connect = t.connectEnd - t.connectStart;
return times;
}
Copy the codeThe log report
A separate log domain name
An independent log domain name is used for log reporting to avoid service impact. First, for the server, we do not want to occupy the computing resources of the service server, nor do we want to accumulate too many logs on the service server, resulting in insufficient storage space of the service server. Second, we know that in the process of initialization page, the page load time, PV, UV, such as data are reported, the report will request and load the business data is almost same time, the browser would amount to a request for the same domain name with the number of concurrent restrictions, such as Chrome, there will be restrictions on concurrency for six. Therefore, you need to set an independent domain name for the log system to minimize the impact on page loading performance.
Cross-domain problems
For a single log domain name, cross-domain problems are definitely involved. Generally, the solutions are as follows:
- One is constructed empty
ImageObject because requesting images does not involve cross-domain issues;
var url = 'xxx';
new Image().src = url;
Copy the code- using
AjaxTo report logs, you must enable the cross-domain request header for the log server interfaceAccess-Control-Allow-Origin:*Here,AjaxIt’s not mandatoryGETAsk, you can overcomeURLLength constraints.
if (XMLHttpRequest) {
var xhr = new XMLHttpRequest();
xhr.open('post'.'https://log.xxx.com'.true); // Report to the node middle layer for processing
xhr.setRequestHeader('Content-Type'.'application/json'); // Set the request header
xhr.send(JSON.stringify(errorObj)); // Send parameters
}
Copy the codeIn my project, I used the first method, which is to construct an empty Image object, but we know that there is a limit on the length of GET requests. We need to make sure that the request length does not exceed the threshold.
Omit response body
For reporting logs, the client does not need to consider the reported results, and even for reporting failure, we do not need to do any interaction in the front end. Therefore, for reporting logs, the HEAD request is enough, and the interface returns empty results, minimizing the waste of resources caused by reporting logs.
Merge report
Similar to Sprite’s idea, if our application needs to report a large number of logs, it is necessary to merge the logs for unified reporting.
The solution might be to try to send an asynchronous POST request for reporting when the user leaves the page or when the component is destroyed, but try to send data to the Web server before unloading the document. Ensuring that data is sent during document uninstallation has been a challenge. This is because the user agent usually ignores the asynchronous XMLHttpRequest generated in the unload event handler because it will already jump to the next page. So is this an XMLHttpRequest request that must be set to synchronous?
window.addEventListener('unload', logData, false);
function logData() {
var client = new XMLHttpRequest();
client.open("POST"."/log".false); // The third parameter indicates a synchronous XHR
client.setRequestHeader("Content-Type"."text/plain; charset=UTF-8");
client.send(analyticsData);
}
Copy the codeThe use of synchronous mode is bound to affect user experience, and even make users feel the browser is blocked. For the product, the experience is very bad. By consulting the MDN document, sendBeacon() can be used, which will enable the user agent to send data asynchronously to the server when the opportunity is available. It does not delay page unloading or affect the loading performance of the next navigation. This solves all the problems of submitting analysis data: making it reliable, asynchronous, and without affecting the next page load. Plus, the code is actually simpler than the other technologies!
The following example shows a theoretical statistical code pattern by sending data to the server using the sendBeacon() method.
window.addEventListener('unload', logData, false);
function logData() {
navigator.sendBeacon("/log", analyticsData);
}
Copy the codesummary
As a front-end developer, you need to be in awe of the product, always pushing the boundaries of performance and intolerance. Front-end performance monitoring and exception reporting are particularly important.
It is possible to add a global exception catch listener for exceptions using window.onerror or addEventListener, but the error may not be caught correctly using this method: For cross-domain scripts, add crossorigin= “anonymous” to the script tag; For code packaged in production environment, the number of lines generated by exceptions cannot be correctly located. You can use source-Map to solve the problem. In the case of a framework, however, you need to bury the exception capture where the framework is unified.
For performance monitoring, fortunately, the browser provides the Window. performance API, through this API, it is very easy to obtain the current page performance related data.
How are these exceptions and performance data reported? Generally speaking, log servers and log domain names are created separately to avoid adverse impact on services. However, cross-domain problems may occur when different domain names are created. This can be done by creating an empty Image object, or by setting the cross-domain request header access-Control-allow-Origin :*. In addition, if performance and log data are frequently reported, the page can be uniformly reported when it is unloaded, while asynchronous requests may be ignored by browsers and cannot be changed to synchronous requests. The Navigator.sendBeacon API came in handy here, as it can be used to asynchronously transfer small amounts of data over HTTP to the Web server. The impact of page unload is ignored.