Recently, the world’s biggest new power car companies can no longer be hot! Xiaobian looked at the skyrocketed share price is really jealous don’t don’t. Those who know the business know that companies led by Tesla have adopted computer vision as the base of autonomous driving technology, and it is through image segmentation technology that the car can tell where is the road and where is the people.

Do I need to emphasize that image segmentation is not important? Today I want to introduce to you the open source suite, including the industry’s most cutting-edge image segmentation algorithms, and excellent results, Paddleseg!! OMG, what are you waiting for? ! Offer him! Offer him! Offer him!

At CVPR2021, the world’s top computer vision conference, Paddleseg once again shone. One of the most influential events in the field of autonomous driving scene understanding in recent years is the Autonue Challenge, which tests the ability of participants in semantic segmentation algorithms in unstructured environments. The Baidu Paddleseg team finally beat the other participating teams and won the championship by ranking first in Level 1, Level 2 and Level 3.

If you are in a hurry, you can go straight to the match details:

https://bj.bcebos.com/paddles…

So what is a PaddleSeg? I went to GitHub to pick up the official explanation:

Paddleseg is an end-to-end image segmentation development kit based on Paddleseg, which covers a large number of high quality segmentation models in different directions from high precision to lightweight. Through the modular design, to help developers complete the whole process of image segmentation applications from training to deployment. Here are some of the Paddleseg’s features and recent updates:

  • A new upgrade of portrait segmentation function, providing a Web end ultra-lightweight model deployment scheme;
  • Introduced the refined segmentation solution Paddleseg-Matting;
  • PanOpTIC – Deeplab algorithm is open source to enrich the model types.
  • EISEG, an intelligent annotation tool for interactive segmentation, was released. It greatly improves the annotation efficiency.

Web video conferencing

Matting

Panoramic segmentation

Interactive segmentation

Provides industry-level deployment. Now there are so many new features. It can be said that Paddleseg has been able to meet the needs of developers in all dimensions in a comprehensive and three-dimensional way. I have to say:

Such a good product, why not get on the bus?

Get on the bus address: https://github.com/PaddlePadd…

Industrial portrait segmentation scheme PPSEG

Portrait segmentation is a very common application in the field of image segmentation. In the practical application, portrait data sets come from a variety of sources. The data may come from mobile phones, cameras, monitoring, etc., and the picture size may be horizontal, vertical or square. Deployment scenarios vary from server side, mobile side, and web side. For this reason, the Paddleseg team has launched a PPSEG model for portrait segmentation trained on large-scale portrait data, which meets the needs of various usage scenarios in server, mobile and Web (Paddle.js).

PPSEG model has been widely used in the industry. Recently, “Baidu Video Conference” has also launched a virtual background function, which enables users to switch backgrounds during video conferencing. The portrait for background model adopts the ultra-lightweight model in the PPSEG series model developed by Paddleseg team. Through Paddle.js, it is deployed on the Web side, and the image segmentation is carried out by using the computing power of the browser directly, and the segmentation effect is highly praised.

Industry level solution a: https://github.com/PaddlePadd…

You can also go to Baidu home page to experience Baidu video conferencing, and directly experience the portrait segmentation function provided by Paddleseg and Paddle.js.

The refined partition solution paddleseg-matting

With the development of segmentation technology, the requirement of segmentation refinement is getting higher and higher. For example, in some film and television industry, green screen as a common work for shooting the background, but the target is not in front of the green screen shooting, whether can also achieve a good background segmentation function?

The answer is: yes!

The Paddleseg team’s recent open source refined segmentation solution, Paddleseg-Matting, addresses this problem well. The target’s hair is precisely segmented.

Paddleseg uses a built-in trimap generation mechanism to make alpha predictions without the need for input of any auxiliary information, greatly reducing labor costs. Reduce the number of network parameters by sharing the weight of encoder, and realize the guidance of alpha prediction of Trimap information flow by using Attention Module in Decoder stage. Then the patch of the misevaluated region was extracted by using the Error Map, and then refined to get the final alpha through the refinement subnetwork.

Interactive segmentation intelligent annotation tool

There is a saying in the industry about artificial intelligence: “Deep learning is as intelligent as the number of people behind it”. This sentence directly speaks to the pain in the heart of deep learning practitioners. After all, the good or bad data of the model occupies a big factor, but the cost of data annotation makes many colleagues in the industry feel headache.

To this end, the Paddleseg team launched the interactive segmentation intelligent labeling software EISEG. What is interactive segmentation specifically? Take a look at the GIF below.

It is not difficult to find that interactive segmentation realizes the edge segmentation of the target object through a series of green dots (positive dots) and red dots (negative dots). The main application direction of interactive segmentation is image editing and semi-automatic annotation, and it can be applied to fine annotation, image matting, auxiliary image post-processing (such as PS) and other scenes.

The PADDLELEG team and PADDLEV-SIG members have launched the industry’s first high-performance interactive segmentation tool, EISEG, based on the RITM algorithm. We support the entire process of training, prediction and interaction with the RITM model. The Paddleseg interactive split model not only supports training powerful generic scene models from scratch, but also supports Finetune for specific scene data. We use Baidu’s self-built portrait data set pair model Finetune to get the portrait interactive segmentation model with fast prediction speed, high accuracy and fewer interaction points.

The software provides a variety of installation methods, support users to use PIP and Conda installation, in addition to Windows provides executable EXE file, double-click. EXE can run the program.

Panoptic-DeepLab for panoramic division

Panoramic segmentation is a new field of image segmentation emerging in recent years. It was first proposed by FAIR and Heidelberg University in 2018.

What is panoramic segmentation?

Image information can be divided into Thing and Stuff. Thing represents countable objects, such as cars, animals, etc., while Stuff represents uncountable objects, such as beach, sky, etc. The semantic segmentation task does not pay attention to whether the image is Stuff or Thing, but only pays attention to the semantic category that each pixel belongs to, so the distinction of instance objects cannot be realized. Instance segmentation focuses on the segmentation of Thing, which identifies the Thing in the image and distinguishes different instance individuals and corresponding semantic information. For Stuff area, it is uniformly represented as the background. Panoramic segmentation is a technology combining semantic segmentation and instance segmentation. For Thing, different instance individuals and their corresponding semantic information can be identified, while for Stuff, the corresponding semantic information can be identified.

Panoptic Deeplab achieves state-of-the-art performance in the form of Bottem-up and single-shot algorithms for the first time. Compared with the top-down Panoptic Deeplab algorithm, it achieves both accuracy and speed with simple network structure, and creates a new direction of panoramic segmentation algorithm. At present, the top algorithm of Cityscape panoramic segmentation is based on Panoptic Deeplab algorithm.

PaddleSeg panorama

The all-star algorithm lineup of 20+ is fully ahead of the high-precision semantic segmentation algorithm of the same framework, and the panoramic segmentation algorithm of 50+ pre-training model is added to enrich the application scenarios. It provides a high precision portrait segmentation algorithm, Humanseg, to meet the multi-terminal deployment.

The deployment of the whole industry chain not only fully supports the development of dynamic graphs, but also smoothly completes the dynamic and static transformation. Also from the data preprocessing, algorithm training and tuning, compression, multi-terminal deployment, the whole process, all links smoothly through, to a great extent to improve the user development of the ease of use, accelerate the algorithm industrial application landing speed. In particular, support for web-side deployment through Paddle.js gives more possibilities for web-side deployment.

What are you waiting for? ! Such attentively research and development of high standard products, also don’t hurry STAR collection car!

Portal: https://github.com/PaddlePadd…