2020.4.25 How to Monitor the Front end


scenario

There are too many components to build the platform. In fact, many functions are similar. How to monitor the performance of components in a class of scenarios

  • The number of references and the number of references indicates the priority of the component
  • Exposure CTR: Lead conversion rate = lead volume/clicks/exposure
  • Data interface: load time, component exception, blank screen rate
  • Configuration complexity: For example, input to select type and default value are preset to reduce user configuration time
  • Code quality: ESlint reports non-compliance during compilation

Monitoring purposes of App and small program:

  • Is anyone using it
  • How is it working
  • What’s wrong with it
  • How to track
  • Analysis to solve
  • Provide decision-making

Tools to song Xiaocai as an example: —- first edition

  • Countly: indicates an RN application
  • Umeng: for native apps

The second edition

  • BugSneg for RN
  • MongoDB data store
  • RGB is built on Koa

The third edition

  • Collect SDK self-developed platform
  • Filebeat Node Node Transfer Collects and forwards logs
  • The Kafka cluster does distributed message communication
  • ES(ElasticSearch) log processing
  • The Datawarehouse sinks into the count bin
  • Bigcousin report platform
  • MySQL/Redis Task storage

System architecture —- Client: PC/H5, RN, small program log processing: log-Transfer, Filebeat, Kafka cluster, ElasticSearch Data processing: Data Warehouse, monitoring system background Data display: Visual report platform, monitoring kanban

Why do you do it yourself?

Take Beibei for example:

Assume that there are 80 projects, and they need to pay about 28W monitoring cost per year > their own research and development cost

Industry scheme comparison (there is a certain error, for reference only) :



Monitoring steps

There are two main scenarios that we can see based on the business,

  • One is used for advertising and marketing, analyzing user behavior to form user portraits
  • The other is basically error monitoring of your own system,

The steps are also different in these two scenarios. User monitoring: buried point -> collection -> calculation -> analysis error monitoring: error collection -> error reporting -> data cleaning -> data persistence -> platform visualization, monitoring error monitoring implementation level from Beibei Allen lecturer has spoken in detail, I have sorted out the brain map here. Web side error monitoring brain map:



The other end



How can the above closed-loop Web monitoring be used in other terminals? In fact, the main application access to the first layer to do the work.

Ideas:Differentiated collection and formatting report

Error capture and code implementation

JS end:

Rejection Tracking: ErrorUtils, Promise.rejection Tracking

Network request: Replace XMLHttpRequest, proxy send/open/onload, etc



Native client:





Wechat applets:

Network request: Proxy wx.request object for global WX



Code implementation:



The React side:



Why ES and Mysql?

Comparison of storage media:





The following is the monitoring and analysis of marketing and advertising scenes:

Buried point

Monitoring user behavior, events, retention -> reported data can be divided into the following levels:

  • Site level
  • The page level
  • Component level
  • Component internal link level

collect

What data to collect

  • User behavior data: user page operations, page jumps, window events
  • Error data: Back-end interface and front-end JS errors AppNative errors

Some of the conclusions we can draw from a single page:

  • Geographical distribution of users
  • The online time
  • Button click rate
  • Page source distribution: internal chain, external chain, search engine
  • Click through rate of advertisement space: the number of clicks/exposure times
  • Daily Traffic Trend

How to collect user stay time?

  1. Use confidence intervals to filter abnormal data. As you can see in the figure below, some average durations are not very good at representing the duration of the user’s stay

  1. A bar chart is used to show the user’s stay duration

How do I collect events? Mouse, scroll, keyboard, custom event concrete implementation:

  • The sendBeacon method ensures that the event is sent
  • Compatible with IE users,Send the request
  • CORS sends a POST request for batch reporting


How to convert the collected data into indicators that people can understand?

Process: set up indicator system -> obtain data -> data analysis -> specific application scenarios



Example 1: User login or visitor access



The three boxes correspond to browser data, event data, and user data. The uUID is automatically generated and associated with the userID, and the UUID and userID are stored in cookies



Example 2: User jump



When jumping, record the source, project ID, page ID, block ID, pit subscript, serial ID,

On the next page, get the UTM value from the URL. Then the page is reported

To calculate

Data cleaning cases:

Analysis of the

How do you analyze it?

Ali Cloud Log Service can be used for user collection, and its main function is query and statistics



Analyze what?





How to analyze basic data PV, UV, click through number, exposure number

Use funnel model to analyze conversion rate

Analyze the page with a thermal map

The conversion rate was analyzed by funnel and path analysis

In the pit of

IOS symbolization



A large number of differences between versions are removed after symbolization, which facilitates error aggregation

Error polymerization

Primary aggregation: full content hash, low degree of aggregation



Hashes errors directly to MD5, so the aggregation is low,

General optimization:

  • The error name and the error stack are the same
  • System class line number removal to reduce stack inconsistencies caused by different system versions (use re to find system classes and remove line numbers)

The information is then hashed

For optimization: the same kind of error, sometimes between different system versions of the client system to solve: extract business code stack and business name to hash OOM error, stack overflow this kind of error, directly get the error name aggregation, and then analyze which devices are prone to occur

Write JSON data to ES

The amount of data is large and all data is not in one index. In this case, an index must be created on a regular or daily basis to save data. At the same time, a fixed index template must be created to ensure that the data type of an index field is consistent; otherwise, data saving may fail. Field issues: Which fields do not change, and which fields change dynamically, need to be measured when designing the system to maintain a moderate degree of flexibility. The changing fields are concentrated in the Pyload information, while other types of fields are fixed, such as:

  • Device characteristics: For example, system type and version
  • User characteristics, such as userId, IP address, and user location
  • SDK information :SDK type and version

Field char processing: If JSON type is reported and JSON data is still saved in ES, new fields will be generated when the fields not taken care of by the template are reported, resulting in an explosion in the number of fields. In this case, log-transfer can be used

Third-party recommendation

  • Their Allies
  • Number of geeks
  • God policy
  • GrowingIO
  • Fundebug
  • FrontJS
  • Sentry

.

review

Because there are things in the back, so today’s activities did not finish, I only listened to some of the sorting. From a business perspective, monitoring scenarios are divided into error monitoring and user behavior monitoring, as well as ali’s more complex component-level, scenario-level monitoring (which is not encountered at present). At the same time learning some monitoring implementation steps, such as burying point, acquisition. Cleaning, analysis is not prior to contact. The content of beibei’s two lecturers is very close to the landing plan. We need to see more PPT in the follow-up, and we will learn more about it if there is demand in this aspect.

About front-end early chat conference: front-end early chat conference goal to become useful, understand, copy to go of the technology conference, planned in 2020 >= 15, jointly held by front-end early chat and Nuggets, front-end early chat conference schedule dynamic, video recording /PPT/ lecture notes download, please pay attention to the “front-end early chat” public account follow up.

On May 16, held the sixth – front end what play Serverless (Paas | Faas), please stamp: huodongxing.com/go/tl6, posters and lecturer itinerary is as follows: