1, the introduction of

Current which intelligent design platform is the user experience design by jingdong retail group online design design service platform, focused on the design of the retail sector, help customers solve encountered in the process of day-to-day operations of various kinds of design requirements, such as commodities to be in the new main photo video, all kinds of festival to promote the commodity main photo update, There are all kinds of design materials needed when the scene with goods is broadcasted.

Linglong is committed to becoming a good design partner for merchants and business operators in the pan-retail field, helping them to realize the digitalization of design, so as to realize the cost reduction and efficiency improvement in design level, and improve the commercial transformation effect of design results (advertising pictures, videos, pages, etc.), which is our vision and mission.

From the aspect of technology and function form, we divide design digital intelligence into two directions, one is “template design”, the other is “programmatic design”.

2. Template design

The core goal of “templated Design” is to realize the digitization of offline design materials, build visual design capabilities such as advertising pictures and short videos on the basis of digital design assets, and provide our end users with a WYSIWYG online design SaaS platform. “Template design” provides customers with three core design capabilities: pictures, animation, video, and commonly used online design AIDS. In addition, through the integration and packaging of capabilities, with the product form of “Linglong Enterprise Zone” for enterprises and institutions to provide a complete number of intelligent design, visual design solutions.

2.1 Image design, so that advertising images in the context of pan-retail marketing are readily available

We provide a lot of fit into the design of the retail vertical templates, as long as customers have a need to sell goods to do design, it can always find needed in antelope which platforms, cooperate with functional experience last iteration “layer editor”, users only need to choose the right design templates use scene, according to the pictures of the wysiwyg simple operation, Can quickly make can be directly used for advertising map; At the same time, it also provides “size expansion”, “intelligent color matching” and other functions to help users enrich the final design effect of the advertising map and meet more use scenarios.

The capability of “Picture Design” also provides PaaS solutions for embedded use. Third party system platforms can use this solution to integrate The visual design capabilities of Telon and build their own business design solutions. At present, the “embedded picture design” scheme has been adopted by several core marketing platforms of JINGdong, such as zhibo, which is used by merchants in shop decoration. By connecting with Linglong’s picture design ability, merchants can efficiently and locally complete the design materials required for shop decoration.

2.2 DYNAMIC graph design, simple two steps to make static graph move

With the gradual popularization of 5G high-speed network and the significant improvement of traffic and speed, the previous strict requirements of Web applications and apps on the size and performance of media materials will gradually become a thing of the past. More and more scenes begin to try to use ANIMATED video to enrich audio-visual effects and improve the commercial conversion rate. Compared to static advertising, giFs require animation expertise and cost several times more to design. Current which platform to provide dynamic graph visualization design, by professional designers will commonly used dynamic effect of precipitation for reusable digital assets, and then through the elaborate design and combination with image template, and the general users can be directly used to provide dynamic graph template, users only need to replace the main figure, fill out the interest point copy, simple two step operation can generate results highlight the dynamic advertising figure, Greatly reduce the design threshold and cost.

2.3 Video design, easy to make short video materials with audio-visual sense

Compared with giFs, videos have more delicate dynamic effects and can also be added with sensible background music, which is the most abundant media material for audio-visual performance and has a high design and production cost. The video visualization design solution provided by Linglong smoothen the cost gap, allowing users to make videos as simple and efficient as making static advertising images. Video visual design solution provides two core editors. One is the background editor for professional designers, which helps them realize the digital management of dynamic materials (GIF, AE animation, frame animation, etc.), while providing flexible layer control, rich dynamic effects and dynamic effects of materials and special sound effects, so that designers can maximize their creativity and create video templates.

The other is the front desk editor for end users. Based on the video templates already designed by designers, users can efficiently produce short videos with good audio-visual effects by combining pictures and copywriting.

The GIF design and video design capabilities provided by Linglong have been widely integrated in jingdong’s internal system platform, just like picture design.

2.4 Design toolbox for you to polish practical picture post-processing tools

Want to do post-processing on existing pictures? Take a look at Linglong’s design toolbox.

  • One-click intelligent matting – Online matting, without PS can also get the bottom map
  • Commodity belt – commodity picture bulk belt, save time and effortless
  • Image batch editing – batch cropping, sizing and compression to format
  • .

2.5 Enterprise zone, so that each enterprise has a complete online design solution

Design for enterprises and institutions to provide several intellectualization can assign of SAAS services, provides including material management, online standards combined figure, quick page set up, the custom component editor, a set of solutions, let the enterprise do not need to put into development costs, can be in the daily operation of the fixed position update and organizing promotional activities such as scenario, Standardized, process-oriented and standardized design output.

Linglong Enterprise Customer – Letxin (www.lexin.com/)

3. Programmatic design

“Procedural design” at the heart of the target is the use of technical means such as data mining, artificial intelligence, combined with the design of the user, for different people to output different style design gimmick, power one thousand thousand surface such as precision marketing scenario, improve conversion rate, used technology including data mining, computer vision, machine learning.

The biggest characteristic of “programmatic design” is “extreme speed” and “unmanned operation”, which is suitable for business scenarios that require a large number of pictures to be generated quickly. A typical application scenario in JINGdong is the focus map of the home page of jingdong APP. It generates different design results for different users by combining user portraits and processes hundreds of millions of image generation requests every day, which is impossible to achieve by relying on manual operation.

Based on the application of “programmatic design” related technical ability, we achieved the basic goal of greatly reducing the cost and improving the efficiency of design. In the past, the designer might only need half a day to complete an advertising picture, but now it can complete several pictures per second in one process of one machine, greatly saving the design cost.

In addition to reducing cost and improving efficiency, the improvement of commercial effect of design results is also a very important goal of “programmatic design”.

We have begun to explore and demonstrate the commercial value of advertising pictures: generate different styles of advertising pictures according to different users’ portraits and design preferences, so as to further improve the commercial click through rate (CTR) of advertising pictures. This new mode of programmatic design based on user’s image and design preference is called recommended design.

Intelligent recommendation design is a complex system engineering, which can be disassembled into several technical issues related to image intelligence. Next, we will give a brief introduction.

3.1 Design portrait

3.1.1 origin

In the construction of recommendation search model, we will construct a large number of labels for users, such as age, gender, city, brand preference, category preference, etc. These labels finally outline the image of a user, which is defined as the digital image recognized by the machine. In the industry, it is called user portrait.

Drawing lessons from user portraits, we began to think about whether users have similar preferences in the field of image vision, and whether people’s aesthetic taste is different because of each person.

3.1.2 argument

As for the answer to users’ aesthetic preferences in design, an article (” A female professor from Zhejiang University finds that the fonts on Coke packaging may be calculating your wallet “) came to a conclusion that impressed us: the use of round and lovely fonts can make users more fond of Coke, and thus make users consume!

This article tells us that in addition to content, design itself seems to affect business conversion rates. We decided to use a series of AB test experiments in jingdong’s actual business scenarios, relying on data to further support its conclusions.

Experiment 1: Study on the influence of different font curvature on CTR

Scenario: on the banner chart of the home page of the core entrance of APP

Some conclusions can be drawn from the monitoring data:

  1. For the category of children or the category of women with a high preference, the rounded font can be considered to improve the user’s liking for the product;

  2. For the category with weak brand recognition, the font with rounded properties can be considered to improve the user’s popularity;

  3. When marketing to female users/young users under the age of 25, more consideration can be given to the roundness of the font to enhance the user’s liking for the product.

  4. Men and women of different ages also have different font preferences

Experiment 2: CTR influence of different colors

Scenario: on the banner chart of the core entrance home page of APP/PC

Experiment 3: Study on the influence of different layouts on CTR

Scenario: on the banner chart of the core entrance home page of APP/PC

Based on the data analysis of the above experiments, we draw a conclusion: there is indeed a preference relationship between different groups in design.

3.1.3 practice

The process of label (preference) data processing is as follows: first, the client reports the user’s operation behavior data (mainly click, browse, search, etc.), then cleans and calculates the characteristics of the data in the hive table in the warehouse, then drops the label into the database, and finally provides the corresponding interface.

Step 1. Data cleaning

The data reported by the front end falls in the data warehouse, the click and exposure traffic model contains the buried data of each business, and we need to clean out the user behavior sequence data for the advertising focus graph!

Example user behavior sequence data model

user Template id time scenario sku_id The exposure number of times Click on the number of times
x_747b7b44d9bc2 01 2020.11.1 The App first focal 23111 12 3
x_747b7b44d9bc2 02 2020.11.1 Pc first focal 21222 12 3
x_747b7b44d9bc2 03 2020.11.2 xxx 2333 4 1
x_747b7b44d9bc2 04 2020.11.2 xxxx 8333 3 0
. . . . . . .

Sample template tag model

The name of the field The field type Enumeration value example
Template group ID ps2id string 5c122d3d82acdd181d18292c
preview url array [‘URL1′,’URL2’]
The scene type scene int 1 – the first focus
Design type design int 1 – the picture; Page 2 –
color color string blue
layout layout string Left, right,
button button string There are
Background style bgstyle string contracted

Step 2. Statistical analysis

Heavy behavioral problems:

When we construct brand preferences for users, we often pay attention to what “heavy” behaviors users have under the brand. “Heavy” behavior can be understood as the user pays relatively high operation cost in order to do something, such as whether the user specially searches for a certain brand of goods.

When we were talking about design portraits, there was almost no way to define this “heavy” behavior, so the usual routines didn’t seem appropriate for building design portraits!

After discussion, we decided to go back to the original statistical approach, assuming that if the user clicks on a certain color, it proves that the user has a certain preference for that color, and then use significance test to verify whether the data is significant, so as to determine the final weight of the tag!

Significance test:

Significance test as a method to judge whether there is difference between two or more data sets has been widely used in various scientific research fields.

Step 3. Label dropping

When we ran out the user tag data, it was really just an engineering problem to put the corresponding data into the corresponding table. However, the actual situation is much more complicated, and there are still problems: the amount of data is too small to cover most users!

Then the method of look alike crowd label is derived, that is, our users may be new users with no data at all. At this time, it is expected to match the labels of similar crowd as the final label result.

Algorithm model

In addition to statistical thinking, we are exploring another way to construct design portraits, using models to train output tags. As I said before, the preference problem can be thought of as a classification problem.

Commonly used classification models mainly include the following two types:

Advantage | | | classification model shortcomings | | : – : | : | : – : | : | : – : | : – | | | decision tree according to the rules of the decision tree can be easily constructed, and the rules are usually easy to explain and understand; The decision tree can be well extended to large databases and its size is independent of the size of the database. Another advantage of the decision tree model is that it can construct a decision tree for data sets with many attributes. | difficulties when dealing with missing data, excessive fitting problems, and ignore the correlation between the data set attributes. | | naive bayesian | in stable classification efficiency. It is good for small-scale data and can handle multi-classification tasks. It is suitable for incremental training, especially when the amount of data exceeds the memory, we can do incremental training in batches. It is not sensitive to missing data and the algorithm is relatively simple, which is often used for text classification. | need to know the prior probability, and prior probability depends on the assumption that a lot of time suppose model can have a lot of kinds, so at some point will be caused by the assumption of the cause of the prior model to predict the effect not beautiful. |

At present, we are trying to implement the tag training output using the decision tree class model XGBoost, which supports various language calls, standalone and distributed, and supports the data format of libSVM sparse matrix.

3.1.4 Outlook and Objectives

Design portrait is a part of the basic ability of intelligent design, combined with the program and map, output different styles of design techniques for different groups of people, help thousands of faces, thousands of fields and other accurate marketing scenes, improve the business conversion rate.

3.2 Real-time combination

We provide an interface for developers and third-party system platforms to quickly synthesize pictures on the server. Developers can build their own design engine according to their own business demands by integrating exquisite procedural design capabilities.

The biggest application scenario of “Real-time combined image” in JINGdong is the thousands and thousands of ads in the first jiao round broadcast of JINGdong APP. Based on the analysis of user portraits and purchasing behavior data without visitors, real-time combined and accurate ads are pushed to improve their business conversion rate (CTR).

Its core is through C to achieve the bottom of the graph engine, and then through the form of NGINX extension into NG, through LUA scripts to control the logic configuration of each business.

In addition, one of the highlights of linglong real-time map combination scheme is that CDN can capture the vast majority of requests, which can effectively reduce the amount of real requests back to the source.

CDN is a kind of accelerated content distribution network. Through the form of multi-nodes, users can access the node resources nearest to users, so as to achieve the technology of rapid presentation of content to users. In short, we can understand it as cache.

3.3 Intelligent color matching

When drawing pictures on linglong website, you can experience the function of intelligent color matching, so as to greatly improve the richness of the picture and switch the picture style with one button.

Based on the image intelligent recognition technology, the color value of the image is accurately recognized, and the intelligent transformation of the color style of the image is realized through the pixel level color value replacement, so as to ensure the style and quality of the color matching results.

3.4 Intelligent Image Matting

Smart matting is built based on jd DrawBot and Meomazhao capabilities. The former is good at product matting, while the latter is suitable for portrait matting. At present, both of these image matting capabilities can be experienced in the small platform, in addition to the interface mode of internal and external enabling.

3.5 Intelligent Layout

Based on the reasoning ability of knowledge graph, we build a set of advertising image layout technology suitable for the pan-retail field. Through knowledge graph, the graph can automatically adapt to the canvas at any size and add appropriate pixel.

3.5.1 Composition of banners of arbitrary size

We developed a method based on knowledge reasoning, from simple to complex recursive migration has realized the banner figure any format of the structure of the composition, using machine learning algorithm to learn a lot of outstanding designers in the template parameter layout, intelligent building accords with the human aesthetic layout composition, using generalization ability of the model implements the format graph ability of any size.

3.5.2 Arranging graphs with arbitrary shapes

In order to increase the richness of the material and the administrative levels sense, we to superposition combination of some basic graphics or text, generate composite material, using a stack convolution algorithm, the primitive generated convolution kernels convolution scanning within the target area, and the target contour area filling, implements the primitive permutation of arbitrary shape contour and superposition effect.

3.6 Intelligent Background

Size expansion is a pain point often encountered in design requirements. An advertising picture often needs to be made in different sizes because it needs to be delivered to different clients. This process we will encounter a big problem, static background images can not be well adapted to various sizes, unlike vector material, it can be arbitrarily enlarged or reduced, and vector background material has a large design cost. Therefore, we hope that we can use the program algorithm to dynamically generate any size good-looking and easy to use background material, it has the characteristics of vector background material, and has a very low production cost, this is the original intention of the intelligent background research, is to achieve AI unmanned design one of the problems, we are now on the road.

There are creative limits to working with machine-generated backgrounds. We feel that the following types of abstract background materials are feasible for machine generation.

3.6.1 Particles + Gradient

Superimposing a large number of particles on a dark gradient creates a background image similar to a sci-fi blockbuster, perfect for electronics. By modifying particle size, color blending mode, randomness and other parameters, you can generate more special atmosphere effects.

3.6.2 Shape combination

The background of pure shape combination has strong universality and can be used in various categories of goods. It is composed of some random shapes generated by algorithms, and a set of color scheme is matched according to the user’s preference style to color the graphics.

3.6.3 Gradient + Decoration

The fusion of decorative graphics and gradient background color is also a common background generation scheme. By modifying the parameters of decorative graphics type, number of layers, fusion mode, position and so on, this background image generation scheme has a strong versatility and can evolve a kaleidanized background material.

3.7 Style Identification

Based on deep learning, we build a predictive model for style recognition, which can identify style feature elements from image information and automatically judge image design style. The technology of style recognition can be applied in precise marketing scenarios such as jingdong APP’s first focus advertising picture of thousands of faces.

3.8 Intelligent color recognition

A color extraction tool, by extracting the RGB value of picture pixels, and then do a sorting, finally through the algorithm from RGB to ordinary people can understand the color (red, blue, yellow, green, etc.).

3.9 Aesthetic evaluation model

Aesthetic evaluation We are in the research stage, and in the future we should combine intelligent generation as one of the evaluation criteria for automatically generated images.

Conclusion – The future of intelligent design

Linglong is currently based on digital design assets, which solves the problems of design efficiency and design cost, but it still relies on the asset input of designers. In the future, we want to realize unmanned design in the real sense. By using big data mining analysis and machine learning and other technical means, we can create a virtual “AI designer”, so that it can make good-looking and effective advertising pictures or short videos just like real designers. The human designer of the future could spend more time understanding the business and thinking about design ideas, leaving much of the design execution to the “AI”.

References:

  1. Image quality assessment model

other

Thank you for your attention to concave-convex laboratory readers. In order to provide better content, I hope you can spare a few minutes to complete a small survey. Next year, the content of concave-convex public account is up to you. Click on the direct

Join the concave-Convex Laboratory open, open source, professional, loving and dream family? Click on the direct

For those readers who haven’t followed bump Lab, please follow us. We only push 4 times a month. We cherish every push and won’t let you down. Wechat search “bump lab” can be followed.

Related articles: Past and future of Bump Laboratory