preface
As developers, we’re all familiar with the term “improve user experience”, and sometimes even ourselves say, “This damn system sucks user experience, can’t it improve the user experience?” But how does it really improve the user experience?
- A new set of cool pages?
- Refactor the code?
- Make the page white screen time shorter?
- Super silky interaction?
- .
Admittedly, these are, but they are partial. Because it’s all about technology to make the system look less bad from the point of view of the system worker. Unless the user has to use your system, there will still be poor feedback when you actually deliver it to the user.
When the version of a product with a large number of users is upgraded, many people will jump out and give feedback that the new version is not good, because everyone has different acceptance of new things. If analyzing whether the user experience is improved from the perspective of some people, it lacks theoretical basis and data evidence. Fortunately, we have a great metric to measure whether the experience is improving: conversion rate.
For a typical site, the number of visitors on the site = the number of new customers acquired by expanding the business + the number of relationships built to cater to existing customers. Both of these can be called conversion channels, and conversion channels determine whether your site gets good traffic and converts more visitors. Conversion rate is the speed at which the site drives conversion. The conversion metric is relatively unique to each site, and for e-commerce, it may be product sales, while for B2B, it may be the generation of qualified leads.
At this point, we can see that the more optimized the channel, the higher the conversion rate, and the corresponding change can be regarded as improving the user experience. One of the most important methods to optimize the website channel in digital marketing is A/B Testing, but not only A/B Testing.
Channel optimization scheme
This part only gives a partial explanation of the scheme. Interested students can browse the relevant materials by themselves.
A/B Testing
A/B Testing is the process of showing two variants of the same page to different segments of the site’s visitors and comparing which variant produces more conversion. In addition, A/B Testing does not mean that it can only be used to compare the quality of two programs. In fact, you can design multiple programs for Testing, such as ABC test. A/B Testing is just A common name.
Core ideas:
- Multiple scenarios are tested in parallel
- Only one variable is different for each scenario
- The fittest are determined by certain rules
Example:
1. Set clear conversion indicators, such as sales volume of e-commerce platform and qualified potential users of B2B mentioned above.
2. Distribute the users according to the variation. Sample the existing users and distribute them randomly (front-end or back-end splitting can be carried out according to the actual situation). Sample calculation formula is as follows:
- N is the sample size required for each group. Because A/B Testing generally has at least two groups, the sample size required for the experiment is 2N.
- α and β are called the probability of the first type and the probability of the second type respectively, generally 0.05 and 0.2 respectively.
- Z is a quantile function of a normal distribution.
- δ is the difference between two sets of values. If the conversion rate is from 1% to 1.5%, then δ is 0.5%.
- σ is the standard deviation, which is a measure of the volatility of a numerical value.
It can be known from this formula that, under the condition that other conditions remain unchanged, the larger the difference between the two experimental groups of values or the smaller the fluctuation of values, the smaller the sample size required.
Of course, we can also use online Tools such as Evan’s Awesome A/B Tools to calculate the required samples.
3. Set the test time and execute the test. Set a test period, such as two weeks, during which the distributed users can only access one of the pushed schemes.
4. According to the test results, analyze the conversion rate and analyze which plan A or B is better through the actual conversion index value, and eliminate the inferior one.
Multivariate Testing: MVT (Multivariate Testing)
This is a way to make changes to multiple parts of a web page and create variations for changing all possible combinations, as shown:
In the figure, the hero picture and CTA color are changed, and four variants appear in the pairwise combination. Multivariate Testing is a sample shunting test of these four variants.
Split URL Testing
Split URL Testing refers to multiple versions of a web page hosted at different urls through traffic sharding. The main difference between Split URL Testing and A/B Testing is that in the case of Split URL Testing, variations are hosted on different urls, as shown in the figure below:
In practical application scenarios, A/B Testing is preferred when basic changes only need to be made on the front end, but Split URL Testing is preferred when major design changes are required and existing web design is not to be touched.
Multipage Testing
Multipage Testing is similar to A/B Testing, except that the changes you make are not made on A single page, but are implemented consistently across multiple pages. Like A/B Testing, site visitors for multi-page Testing can be divided into one version or another. By tracking how these visitors interact with the different pages that are displayed, you can determine which design style is most effective.
There are two ways to perform multi-page tests. First, you can use all of your pages as a base overall funnel, then create multiple variations of it, and then you can test the new variations against the base funnel. This is called the funnel multi-page test. Second, you can test how the addition or removal of duplicate elements (such as security badges, personal identification, etc.) affects the transformation of the entire funnel. This is called a “classic or regular multi-page test.”
With the above optimization, it is possible to use statistical methods such as frequency theory or Bayesian to accurately analyze the user experience of this variant, as well as to predict indicators such as the acceptance of the system by new users.
conclusion
The first object to improve user experience should be “users”. Moreover, strong data from the “users” side should show that this variant has a positive impact on users.
One last question: how do you measure an improved user experience in a mid-background application?
Author: ES2049 / Merlion
Figure source from the network, if there is infringement, please contact to delete
The article can be reproduced at will, but please keep this link to the original text.
You are welcome to join ES2049 Studio. Please send your resume to [email protected].