background

Xianyu is a typical idle trading platform in C2C scenarios. Every user in Xianyu can enjoy the fun of free trading by simply inputting the commodity name, price, inventory and other information to complete the release of a commodity. Full freedom is the source of xianyu’s vitality, but it is a challenge for an e-commerce platform. The commodity structure that this article wants to say is one of pain point.

The reason why commodity structuring on C2C platform is difficult is that users are not motivated enough to complete the structuring. How to let users with the minimum cost to complete the structure, we can not hope in the business background so heavy solution, we need a simple, efficient and flexible solution.

Scheme selection

How to solve it? First of all, we make a comparative analysis of a scheme from the whole cycle of C2C commodity release.

Idea one offline solution

The modification scheme includes algorithm association & socialization scheme. The algorithm association scheme is to analyze the commodities published by users through technical means, so as to carry out the association or attribute marking of the same style. The core of the socialization program is to package the commodity into a structured activity. This can be through the user to participate in the way of answering questions, the association of structured commodities. The disadvantage of the offline solution core is that the link is too long and the data backflow is slow. The more important problem is that the analyzed data cannot be used in the display domain without user confirmation.

Train of thought second-hand motion associated scheme

This is the process in release. This is probably the most intuitive solution. In the publishing process, guide users to do attribute marking or the association of the same type of goods. The advantage of this scheme is simplicity and intuition. The downside is equally obvious: shifting costs entirely onto users. For sellers of C, each additional release option may lead to user loss. This scheme can be used as a structural supplement, but is not the optimal solution we are looking for.

Our thinking: Can we achieve an efficient and low-cost solution under the premise of ensuring real-time performance?

The answer is the intelligent publishing solution presented in this article. If xianyu users are in the release stage, they can automatically associate the goods to be released with a certain item in the goods library of Chaotao. The commodity can use some structured information of the same commodity, and the problem of commodity structure is solved.

Scheme comparison

It can be seen that intelligent publishing scheme is the most balanced scheme with cost and effect.

The business logic

First, let’s understand the core logic of the product (take video release as an example).

Simple disassembly:

1. Focus the subject

As the beginning of intelligent identification. We need to use the AI algorithm to identify the subject object of the end-to-end shooting. The goal is to be consistent with the user’s perception of the object.

2. Intelligent recognition & guidance

We will carry out real-time identification of the photographed object in the user’s shooting process. At the same time, we guide the user to shoot the effect of the core information amplification algorithm of the target object.

3. Result feedback & user confirmation

When the user finishes shooting, we will give the user a simple choice in the same way.

Architecture design

Technical challenges

To sum up, the core solution of intelligent publishing is to transform the product problem of product structure into a technical problem of matching the same item.

So our core technical challenge:

  • Real-time guarantee of commodity identification in the release stage

  • How to maximize the matching success rate of the same item by squeezing the AI’s power

To solve this problem from the perspective of idle fish, we have three technological advantages:

  • Mobile AI solutions represented by AliNN enable end-to-end AI computing

  • We have probably the largest commodity information base on the planet (Taobao & Tmall)

  • Ali Dharma courtyard strong AI capabilities

We can greatly improve the real-time performance of the link through the front part AI capability to the end side. At the same time, we combine AI recognition ability with Amoy commodity database to complete the function of matching the same commodity.

In order to achieve the above capabilities, we have built a complete intelligent publishing technology architecture.

The overall architecture

Let’s start with our logical architecture

The overall design is divided into three layers:

  • UI presentation and interaction layer. The core is to handle user input and result feedback.

  • Logical processing layer. It mainly controls the operation logic of intelligent pipeline identification and distribution of the results of sub-module processing

  • The framework layer is mainly composed of core processing sub-modules

Architectural details

In detail, we build efficient recognition by combining three layers of flutter, Java /Oc and C++ logic. As shown below:

Main design considerations:

1. Make full use of different technologies to maximize r&d efficiency.

We developed using Flutter at the UI layer to take advantage of Flutter’s multi-terminal consistency. We also sank some of our common algorithms into the C++ layer. This can greatly improve the reuse rate and consistency of logic at both ends.

2. Make full use of the computing capability of the end

Fuzzy detection, similarity detection, subject recognition, tracking these algorithms are implemented at the end. In addition to making full use of end-side computing power, it is more important to improve processing efficiency in the shooting process. Minimize dependence on network requests.

Through the extreme compression algorithm, the size of the final uploaded picture is controlled at about 10K. Even 4 requests is only 40K. It can be said that there will be no pressure on user traffic.

3. Pipeline arrangement system

Considering the continuous optimization of the later system, the adjustment of the processing logic of the sub-module is inevitable. So we designed a flexible pipeline to manage all the processing logic. The pipeline can flexibly combine Java /Oc and C++ capabilities. And it is convenient to adjust the order of sub-functions and add or subtract functions. The architecture design is as follows (take Android as an example) :

4. Protection of user privacy

Images used for identification are encrypted to minimize the risk of user privacy breaches. Image addresses that appear in the public domain are not directly accessible. Even if the user’s privacy can be protected.

Algorithm architecture

We also did a lot of optimization on the algorithm side.

The core algorithm of intelligent publishing is the matching algorithm of the same goods. We improve the prediction algorithm of single frame to multi – frame prediction. And we innovatively make deep integration of algorithm and interaction, and squeeze the ultimate ability of algorithm. The process is as follows:

If the algorithm finds that the current frame is not enough to make a more accurate algorithm prediction, the image information will be passed back. In the transmission process, users are guided to take the information needed by the algorithm through copywriting in time. Iterate until the product information is completely predicted. The algorithm processing logic is shown in the figure below.

The effect

Real-time processing performance: through our test, the recognition process has no obvious perception to the user except the active prompt. There are no performance problems such as frame loss during the normal shooting process.

The recognition effect of the same product: overall, the accuracy of multi-frame recognition is improved by about 20% compared with that of single frame.

Through this project, we not only built a complete real-time recognition capability of idle fish for commodities. At the same time also precipitation image pretreatment, tracking and a batch of terminal computing core algorithm. Based on this, it’s possible to give more real-time AI capabilities to more scenarios (such as scanning specific products or participating in specific events with logos).

Looking forward to

Smart release will be released in September, we welcome your feedback. The first part of the video release will be online, followed by pictures, activities and other scenes. Through the intelligent identification of the project, we believe that we can continue to improve the free fish commodity structure rate.

The future release we envision is a highly intelligent release. Based on the deep understanding of the product based on the camera, the system directly gives the product information, structured labels, recommended prices, and even the degree of old and new. All the user has to do is confirm. Today’s smart launch is only the first step in our great journey, and we will continue to work towards our goal!

The Idle Fish Team is the industry leader in the new technology of Flutter+Dart FaaS integration, right now! Client side/server side Java/architecture/front-end/quality engineer all look forward to your joining. Base Hangzhou Alibaba Xixi Park, we will work together to create community products with creative space, do top-level open source projects in depth, expand technological boundaries and achieve the ultimate!

* Send resumes to small idle fish →[email protected]


More series of articles, open source projects, key insights, in-depth interpretation

Please look for the idle fish technology