Author: RichLab Front-end Building

Welcome to the technical column of “Huabibaibi Front End Team”, we will share with you high-quality technical articles in various fields of front end, including but not limited to mobile terminal, small program, interactive technology/data visualization, Node.js full stack/middle background, infrastructure, personal thinking, not limited to original and translation. We also welcome submissions of quality articles.
Editor’s note: This is a long article about 3D interactive development. It will start from the basics and explain ant Financial’s exploration road in 3D interactive field from simple to deep. If you’re interested in interactive technology, don’t miss this article!


Image, as the visual basis of human perception of the world, is an important means for us to obtain information, express information and transfer information in this information age. The most efficient and accurate way to generate images is to be generated, displayed and drawn by computers. These technologies are also referred to as computer graphics technology. Computer graphics technology has been the technical foundation of many industries, such as animation, video effects, games, design, advertising, AR, VR, data visualization and so on. In order to get closer to the real human experience and enhance the sensory experience, these industries will preferentially choose 3D images. Like the ones we know.

(Picture from Internet)



In recent years, computer graphics technology has been the focus of research at home and abroad. It is a very deep field, and there are many problems to be solved whether it is academic in-depth research or content industrial output exploration. Among them, games, movies and other fields have developed relatively mature in this respect, but the computer graphics technology on the Web side has just started.

Ant Financial has always attached great importance to interactive experience, so this area is also our key research direction, especially the computer graphics technology on the Web end. At first, we simply applied computer graphics technology to serve our products. Later, we gradually participated in the exploration of this field and formed a complete solution in terms of basic capacity building, tool platform, workflow and implementation scenarios.

Next, I would like to share with you the problems we encountered in the process of exploration and the corresponding solutions. Welcome to discuss and exchange.

First, let’s take a quick look at some pre-knowledge.

Real-time graphics rendering

In fact, 3D image rendering is divided into offline rendering and real-time rendering.

Offline rendering pays attention to the sense of reality of the image and does not have harsh requirements on calculation and time. It may require hours or even days of computer operation to render a frame of image. It is usually used in animation films, advertising illustrations and other fields with high quality requirements.

Real-time rendering is very computationally time-critical. In human-computer interaction scenarios, the screen needs to be refreshed in real time with the user’s operation and control, and smooth animation is drawn and fed back to the user. This requires the computer to render an image in a short amount of time and display it in real time, followed by the preparation of the next image. It is difficult to complete real-time rendering only relying on THE computing power of CPU, but also needs the parallel computing power of GPU. Real-time rendering is mainly used in human-computer interaction scenes and rendering scenes showing different images according to different data, such as games, data visualization, AR, VR and so on.

In the World of the Internet, most of the scenes we deal with fall into the category of real-time rendering. Although the real-time rendering done castration on the rendering, but we believe that with the development of the technology, the real-time rendering effect will be more and more realistic, previously available only through offline rendering can achieve the effect of the future can be real-time rendering, virtual reality and augmented reality in the future will be a time and again shake our eyeballs.

Web side 3D image rendering

Rendering 3D images efficiently depends on the control hardware and graphics rendering interface. In order to make browsers capable of rendering 3D images efficiently, WebGL working group released the WebGL specification in 2011, which was soon implemented in all browsers. With WebGL, gorgeous 3D effects can be made on the Web side. With the cross-terminal, cross-client and flexible and dynamic features of The Web, users can obtain and load 3D content at low cost without downloading 3D application client. At the same time, It can also be combined with product content in super apps such as Alipay at a low cost to take full advantage of its traffic.

Web-side features combined with 3D rendering have changed many user scenarios, such as:

  1. Taobao commodity display area can show users a more intuitive 3D commodity model, enabling users to browse commodities in an all-round way.
  2. 3D decoration design is a one-time demand, no longer need to spend time to download a professional software, open a browser can also get a panoramic experience.
  3. 3D mini-games produced on the Web end, almost no download time, users can try at low cost, no obstacles to spread, easier to match target users.
  4. Maps in the browser can be rendered in 3D, models loaded on demand, and dynamically rendered.

(Autonavi 3D Map)



There are also a number of 3D applications on the Web side, which are not listed here.

Some scenarios don’t sound complicated, but it’s not easy to do them directly with WebGL. WebGL itself only provides the most basic graphics rendering API, so you need to fully understand the internal details of WebGL, learn how to do GPU programming through shader language, and solve problems such as spatial relations and light and shadow effect calculation with complex mathematical knowledge.

Therefore, we hope to encapsulate an application developer-friendly engine based on WebGL, providing a large number of commonly used apis, and simplifying the whole 3D project construction process. Oasis 3D was born.

Oasis 3D

Oasis 3D (Oasis ENGINE, hereinafter referred to as Oasis 3D) is a lightweight 3D engine developed by Ant Financial. It provides rich functional components and supports complex 3D graphics rendering. Developers do not need to learn complex graphics knowledge. Oasis 3D makes it easy to create complex 3D scenes, greatly reducing the difficulty of creating 3D scenes.

In the field of Web-side 3D engines, Oasis 3D has the following features:

  1. Microkernel architecture. The Oasis 3D kernel provides the basic rendering capabilities, but is not limited to that. You can extend Oasis 3D to different engines for different scenes, such as a game engine, an animation engine, a data visualization rendering engine, an AR scene 3D rendering engine, etc. Oasis 3D absorbs Unix ideas. Every module of Oasis 3D follows the concept of “Do one thing, and do it well”, completing complex and cohesive function sets in the field, exposing only the most understandable and simple interfaces, and combining powerful and complex whole. It does not affect the maintainability of the overall function. At present, Oasis 3D is mainly applied to interactive games and interactive large screens in ant Financial system. In order to support these scenes, Oasis 3D encapsulates commonly used graphics function modules and basically supports our daily application development:
  • SceneGraph scene structure. SceneGraph can be used to manage complex scenes and spatial relations between scenes in graphics applications.
  • 3D rendering basic functions: light source, mesh, geometry, material system, camera;
  • Rendering enhancements: Shadows, post-processing system, scene atomization;
  • External model support glTF 2.0 and matching PBR material;
  • Rich and diverse animation systems: bone animation, interpolation animation, particle animation system;
  • A series of powerful functional components: collision detection, HUD parallel display, ray projection, frame buffer pickup, finite state machine, math library, and more;
2. Component-based development. Component-based development (CBD) and traditional object-oriented programming (OOP) can both be used to extend the functionality of a class, but there are very different design approaches. In traditional object-oriented programming (OOP), a base class is designed so that when features need to be extended, derived classes are generated by inheriting from the base class. This approach is straightforward, but it can be very problematic as the number of features that need to be extended increases. For example, if you have a base Animal class called Animal, if you have an Animal that can fly, you have to make an extension called FlyAnimal. If there is an animal that can swim, the extended SwimAnimal class is generated. But what if there was an animal that could both fly and swim? First of all, you can’t inherit FlyAnimal and SwimAnimal at the same time because both classes have a common base class; Second, if we were to generate the FlySwimAnimal class based on the Animal extension, we would have to rewrite the FlyAnimal and SwimAnimal code twice. Component-based development (CBD) does not have this problem when it comes to extending functionality. In the example above, we have an Animal base class called Animal, which can implement the FlyComponent if we want to extend the ability to fly. To extend swimming capabilities, implement the SwimComponent. To use it, just add the required components to Animal. If you have an animal that can fly and swim at the same time, simply add FlyComponent and SwimComponent components to the animal to extend it. Here are two diagrams of the structure of object-oriented programming (OOP) versus component-based development (CBD) programs:

(OOP vs. CBD)

It is not difficult to see that in 3D applications, component-based development (CBD) is more suitable for function expansion than object-oriented programming (OOP), with simple and clear structure. At the same time, the code coupling caused by inheritance is greatly avoided. This is great for visual development.

3. Mobile friendly. The Oasis 3D itself is lightweight and can be loaded on demand, even when mobile network speeds are limited. Oasis 3D has also made a number of optimizations for mobile browsers, resulting in some best practices.

Having experienced the baptism of ten million business of Ant Financial, Oasis 3D has been adapted to thousands of models on mobile terminals, achieving stability and reliability.

To sum up, Oasis 3D, which is more suitable for the 3D engine model and thinking on the Web side, takes more consideration for the mobile environment and has an advantage over other engines on the market.

Project course

Ant Financial has been exploring the implementation of 3D projects in recent years. Oasis 3D has gone through the training of one project after another, continuously iterated and optimized, summarized practical experience, and gradually formed an efficient 3D workflow.

When Oasis 3D was released in 2017, the game of Ant Manor and Star Ball was successively adopted. , the second anniversary of the bank of Online commerce particle array special effects, flower bai Double eleven special effects, borrow bai Bai shell whale special effects and other projects.

(Ant Village – Star Ball Game)

In 2018, one of the things Oasis 3D did well was to optimize the API, release a new version, set up an official website and promote it within Alibaba Group. We also explore the mobile side projects, such as the hui planet, ant farm chicken mountain climbing and so on, more and more adopted mobile friendly Low poly real style to carry on the design, use a lot of baking textures instead of complicated illumination and shadow, so, visual effect and performance are greatly optimized, the user experience is improved significantly.


(Huizheng)

(Ant Farm – Chicken Climbing Game)


At the same time, we also made a try in the visual large screen project, and successfully completed the important task of undertaking the company’s business large screen.

(Alipay large screen)



The Oasis workflow

Although Oasis 3D made its debut as a 3D project, the whole research and development process was quite difficult, and a series of problems in the production of Web 3D projects were exposed in the process:

  • Too much customization. Especially for mobile terminal projects, the rendering ability is greatly limited by the device. In the case of not affecting the effect, but also to achieve smooth operation, so some customized solutions are often used to achieve a clever way. This requires users to master the principles of Oasis 3D engine and do some customized development if necessary. The overall development is difficult and inefficient.
  • The cost of getting started is high. Front-end engineers are not familiar with computer graphics technology and game workflows, resulting in a small number of engineers who can handle 3D projects.
  • Less precipitation, creative difficult. Most designers of the product line are not good at 3D design and do not know how to design, or the designed scheme may not be implemented. Designers need to understand the characteristics of the actual operating environment for mold design, and it is best to directly expand according to the effect supported by the current environment. Because there are few cases of 3D precipitation on the Web side that can be referred to at present, it also increases the difficulty of landing 3D projects.
  • Cumbersome workflow. The process of converting the model made by model art into high-quality runnable resources is tedious, and it also needs to use Unity for configuration and conversion, resulting in heavy workflow.
All these problems are obstacles to the implementation of 3D projects on the Web side, which makes it difficult to popularize and reuse this technology in ant system.

So, not only do we have to solve the engine level problems, but we also have to make it easy to land a Web 3D project and make it accessible to as many people as possible. This forced us to build engine functions, tools and workflow, and platform ecology at the same time.

So we developed a 3D content editor based on Oasis 3D
Oasis Editor.

Oasis Editor is an online Editor that redefines the workflow of 3D projects. With the workflow provided by Oasis Editor, you can visually edit 3D scenes and interactive logic. Functions including scene building, scene beautification, animation editing, and game logic editing can be completed online.


(Oasis Editor)


The overall Oasis Editor has the following advantages:

  • Rich functional components. Oasis Editor supports a rich set of resource types that extend custom requirements to new functional components through custom extension capabilities.
  • Art resources are seamless. The Oasis Editor automatically converts and uploads most OF the 3D resources, eliminating all Unity conversion work.

(Comparison diagram of the docking flow of different art resources)

  • All kinds of animation visual editing. Oasis Editor provides a variety of animation editing functions for engineers and designers, such as interpolation animation editing and particle animation editing. This allows us to move away from the workflow of repeatedly adjusting parameters in code and then showing them to designers. Designers can adjust parameters directly using the Editor. Get involved in the development of projects – leave the professional work to the professional people.
  • Online platform ecology. Oasis Editor works with a 3D content platform, providing project iteration management and resource sharing capabilities.
In addition, the Oasis Editor tries to cover every aspect of our 3D development. For example, in computer graphics technology, the most specialized part is the use of various graphics algorithms and programmable shader interface to achieve various effects and simulation effects, application development engineers actually difficult to get involved in this part of the work. However, many specialized shaders are reusable, and visual shader editors and shader content management platforms are also important. In the future, we plan to integrate this function with Oasis Editor project, so that we can not only show users all the shader effects we support, but also reduce a lot of customized development work.

In general, we hope that Oasis Editor can simplify the tedious and difficult work in 3D development process, so that developers have no awareness of Oasis 3D engine, Oasis workflow can complete the development of the entire project, so that 3D development ability is at your fingertips.

conclusion

Browser in order to achieve the basic commonality, provide to the user is the most basic drawing interface, users can complete freedom to each content on the canvas control point according to, but too much freedom is a double-edged sword, your imagination and knowledge of graphics system completely limit the can you apply colours to a drawing gives what kind of work, and most people don’t have such professional ability, This makes it extremely difficult to land 3D projects on the Web side at present. It’s as if we wanted to build a skyscraper, but the browser only provided the most basic sand and gravel.

Like building buildings, our 3D projects on the Web side must also gradually become industrialized. The Oasis architecture does away with the underlying expertise, providing users with tools that are easy to use and a rich platform.

We have been working hard on the construction, and soon Oasis system will be open to users outside ali system, providing services for all Web 3D project developers, please look forward to it!


Finally, thank you for reading such a long article carefully, throw an egg ~

🎉 our team will also produce a series of articles in the field of 3D technology, interested partners please pay more attention to oh ~



The front end team is committed to sharing high quality technical articles with you

Welcome to follow our column, share the article with your friends, and grow together 🙂



Our team is urgently recruiting: interactive graphics technology, front-end/full stack development, front-end architecture, big data development and other directions, the expectation level P6+ ~ P7, the team technical atmosphere is good, big room for advancement, resume can directly hit me [email protected]