background

In the promotion of business, we often encounter the composite two-dimensional code poster sharing function, and in order to promote, we need to have this function to increase exposure in APP, WEB and small programs at the same time. Each end needs to be written separately, with poor reuse ability and low efficiency. It is not difficult to synthesize posters by itself. In this context, in order to improve efficiency, we developed the Lumu-Poster synthesis tool (technical stack: NestJS + React + mysql).

Analysis of the

There are many mature solutions for poster generation itself in the existing community, as follows:

  1. Html2canvas/Canvas plugin screenshot
  2. The server (Java, Node, etc.) is drawn
  3. The server is generated using the Puppeteer headless browser

Html2canvas/Canvas drawing screenshot

  • Advantages: completely free server, independent generation by the front end, custom style strong.
  • Disadvantages: some mobile phones lack compatibility, cross-terminal reuse ability, performance depends on the mobile phone itself

Server plug-in Drawing

  • Advantages: No front-end processing is required, strong cross-end reuse capability and excellent performance.
  • Disadvantages: lack of ability to customize styles, relatively complex coding

The server is generated using puppeteer

  • Advantages: Strong customization and reuse ability.
  • Disadvantages: Insufficient performance upper limit

Because our posters are not only pictures, but also have partial personalized contents such as tables and long graphs, we value personalized extension and cross-reuse ability, and finally choose puppeteer to generate by comparing the above scheme.

Puppeteer pain points

The community itself has plenty of scenarios and articles about puppeteer generating images, but several problems have been identified

  1. The community uses puppeteer’s common urls to access pages and generate images, resulting in the need for a front end to create a page for each composite image and connect to dynamic data, putting all the work on the front end.
  2. In the existing poster scheme, all the network pages are loaded through page.goto, and each poster needs to create a page, test, deploy and publish; The process is complex and does not liberate the front end. The response speed is particularly dependent on page resource loading and network state. In the case of separation of the front and back ends, the dynamic loading content required by the page request in the background is about 800ms-1500ms, and the simple screenshot page rendered by the server is basically optimized to 400-500ms. Even so, with other business processes, the interface response is about 800ms.
  3. Puppeteer needs about 30M memory for each TAB page opened, and the CPU load caused by opening multiple tabs at the same time determines the upper limit of puppeteer stand-alone and the code optimization is limited.

The solution

How do I solve these problems for the three pain points mentioned above?

About pain point 1

It mainly needs to liberate the front hands. The poster business itself is not complicated, generally pictures (background picture, head picture, TWO-DIMENSIONAL code, etc.), words, tables and other simple elements. This kind of simple and specific business can be generated in a visual way. Here, I choose React as the UI framework and use customized JSON scheAM as the data storage format to dynamically render the page. Inherent in design visualization are component choreography and form choreography.

Component layout

When designing the layout of components, considering the availability of visualization in other projects, the plug-in method is used here to enable the components, and the components themselves only have rendering ability. If it is necessary to extend the capabilities of the components and use the way of high-order components for injection, the absolute layout is used uniformly here. Drag and drop is implemented using the Moveable plug-in for secondary encapsulation into a higher-order component.

Json scheAM definition of the underlying component on which all components need to be inherited

// Component type
export type ComponentTypes = 'TEXT' | 'PICTURE' | 'CANVAS'; 
// Component configuration
export interface ComponentSchema {
    /** Unique id of the server */
    id: string;
    /** Node name */
    nodeName: string;
    /** Component type */
    name: ComponentTypes;
    /** Component outer mount ID */
    domId: string;
    /** Combination id */
    groupId: string;
    / * * * / parent id
    parentId: string;
    /** Component style */
    styles: React.CSSProperties;
    /** Custom content */
    custom: unknown;
    /** Subcomponent content */children? : ComponentSchema[]; }Copy the code

An example text component

import React from 'react';
import { ComponentSchema } from '.. /.. /interface';

export interface TextProps extends ComponentSchema {
    custom: {
        text: string}}const Text: React.FC<TextProps> = ({ custom, styles, domId, groupId, children }) = > {

    return (
        <div
            className={groupId}
            id={domId}
            dangerouslySetInnerHTML={{__html: custom.text}}
            style={styles}>{children}</div>
    )
}

Text.displayName = 'Text';

export default Text;
Copy the code

Then the components are registered through the component plug-in system, and specific high-level component injection is carried out during the registration. The process is as follows:Page rendering component

import React from 'react';
import { ComponentSchema } from './interface';
import { getViewPlugin } from './view-plugins';

const StaticViewRender: React.FC<{ dataSource: ComponentSchema[], mock: boolean} > =({ dataSource, mock }) = > {
    return (
        <React.Fragment>
            {
                dataSource.map((data) => {
                    const plugin = getViewPlugin(data.name);
                    if (plugin) {
                        const Component = plugin.component;
                        return (
                            <Component key={data.domId} {. data} mock={mock}>
                                { data.children && <StaticViewRender mock={mock} dataSource={data.children} />}
                            </Component>) } return null; })}</React.Fragment>)}Copy the code

The form layout

In the visual system, forms mainly generate corresponding forms through component attributes, and we do not want users to edit some attributes. Therefore, when designing forms, we also define forms by custom JSON Schema and inject them into the system through plug-in registration, and associate components by component names. The normalization of antD forms makes it easy to write basic dynamically rendered forms. Because the poster itself will have dynamic content, a simple string parsing is done to inject dynamic data through rules. When creating the poster, the editor is told by ${variable} that this is the key of the dynamic data, and when rendering, the user calls the match by url query. After completion of such as poster will never a url http://x.x.x.x/tool/screenshot? Id =10&clipWidth=694 &clipheight =684&name= &headimg =&code= This URL is the image address, where name/headImg/code is the dynamic data key when making the poster

Effect:

In this way, the front end is basically liberated. Generally, posters involve 4-5 elements and can be released within 3 minutes.

Pain point 2

The visual editor has completed the page generation, now through direct access to goto way, but found itself system use the react, react based component package itself a lot and if more and more packaging resources will increase, and the modern way of development, needs to be done by js read again after the past of the current template json, then apply colours to a drawing, This leads to a direct drag in rendering time. At first, I thought of using dynamic rendering components to reduce component resources. However, I found that there were not many component resources in the poster business, and react itself was the main resource. At that time, I thought that using React for poster rendering was overuse. Completely avoids the performance cost of JS rendering. The react and ReactDOM packages are not small enough to render HTML, but they are also used to render HTML. React Vue is a bit of a overuse, but the setContent API in Puppeteer can be directly injected into HTML for page rendering, which can avoid web factors to the greatest extent. We don’t need JS and CSS for posters. Feel the scheme is great, that how easy to inject HTML? In fact, when we do the visualization, we already have HTML, but there is no dynamic content, so we can use the rules to store the HTML directly into the database when we are done. Through this set of rules, poster synthesis can be basically completed in 150ms-400ms, reducing package volume and network factors to the greatest extent.

  @Get('/updateScreenshot')
  async updateScreenshot(
    @Res() res: Response,
    @Query() query: {
      id: string,
      clipWidth: string,
      clipHeight: string, clipType? :'png' | 'jpeg', clipScale? :string,
      [key: string] :string
    }
  ) {
    const { id, clipWidth, clipHeight, clipScale = 1, clipType = 'png'. params } = query;const pictureInfo = await this.pictureService.findById(+id);
    // Inject real data by parsing custom rules
    const html = renderStr(pictureInfo.html, params);
    const buffer = await pool.use(async (page: puppeteer.Page) => {
      await page.setContent(html);
      const buffer = await page.screenshot({
        type: clipType,
        encoding: 'binary'.clip: {
          x: 0.y: 0.width: +clipWidth,
          height: +clipHeight
        }
      }) as Buffer;
      return buffer;
    });

    res.setHeader('Content-Type'.'image/png');
    res.setHeader('Content-Length', buffer.length);
    res.status(200).send(buffer);
  }
Copy the code
const renderStr = (str:string, context: object = {}) = > {
  const tokenReg = / \ $(\ \)? \ {([^ \ {\} \ \] +) (\ \)? \}/g;
  //@ts-ignore
  return str.replace(tokenReg, (word, slash1, token, slash2) = > {
    if (slash1 || slash2) {
      return word.replace('\ \'.' ');
    }

    let variables = token.replace(/\s/g.' ').split('. ');
    let currentObject: object = context;
    let i, length, variable;
    for (i = 0, length = variables.length, variable = variables[i]; i < length; ++i) {
      //@ts-ignore
      currentObject = currentObject[variable];
      if (currentObject === undefined || currentObject === null) return ' ';
    }

    returncurrentObject; })}Copy the code

About pain point 3

For the time being there is no particularly good solution, we can use serverless or we can use queue to smooth the output under high concurrency.

The whole process

Graph TD visual drag and drop to generate the page --> save the HTML and other information according to the page --> The server analyzes and infuses the data according to the query parameters and obtains the corresponding template HTML --> infuses the puppeteer with setContent and returns the screenshot to the user

The last

I have made two NoCode editors, and FOUND that NoCode cannot be large and comprehensive. Instead, it is targeted at specific scenes or subdivided fields. Only in these scenes, the front-end interaction is relatively simple, so enough components can be deposited. No matter what kind of scene the bottom layer needs to rely on component choreography and form choreography, the development of these two types becomes particularly important, I hope to get to know more friends who are interested in NoCode/LowCode/ProCode and communicate with each other.

Additional: can also take a step further, by uploading PSD file parsing into components, so that even drag can be directly saved, to achieve second generation posters.