About & Introduction

The content of graphic rendering occupies an indispensable part in the whole knowledge system of the front end. Whether it is data visualization, 3D model display, or H5 game development, it is necessary to have an understanding of the knowledge of graphic rendering.

This paper starts from the front-end perspective, to do some basic knowledge of the introduction to graphics rendering, and actually finish rendering a simple 3D model in the browser.

Apply colours to a drawing

There are many data models and mathematical algorithms in the process of code implementation. In order to avoid affecting the overall learning process by throwing too many concepts before the implementation, the explanation will only be expanded when a certain data or algorithm is used.

2D color block rendering

Almost every 3D model is composed of many triangular faces (even if some output models are not triangular faces, a polygon face can be converted into multiple triangular faces, that is, triangle is the simplest plane, and the combination of many planes forms a model).

Every triangular face is composed of three vertices, in other words, a model is composed of multiple vertices, so in the rendering process, our core focus is the change and rendering of vertices.

Below is an example of how to render a simple triangle.

Function renderPlane() {const canvas = document.getelementById ("renderCanvas") as  HTMLCanvasElement; const ctx = canvas.getContext("2d"); // Canvas width determines the number of pixels to draw const height = canvas.height; Const v1 = new Vertex(); const v1 = new Vertex(); V1. Position = new Vec4(0, 0.2, 0, 1); const v2 = new Vertex(); Position = new Vec4(-0.2, -0.2, 0, 1); const v3 = new Vertex(); V3. position = new Vec4(0.2, -0.2, 0, 1); /** Viewport transform: Map the standard plane to the screen resolution range, i.e., map the coordinates of [-1, 1]^2 to [0, width]*[0, height] in the canvas environment, Width & height = canvas.width & canvas.height simply means that a point in one coordinate system is mapped to another coordinate system */ // convert to viewport coordinates, Const viewPosition1 = getViewPortPosition(v1.position); const viewPosition2 = getViewPortPosition(v2.position); const viewPosition3 = getViewPortPosition(v3.position); /** 1 Uint8ClampedArray imageData: {data: {0: 255, R value 1: 0, G value 2: 0, B value 3: 255, A value}, colorSpace: "SRGB ", color type height: 1, width: 1 Uint8ClampedArray */ const imageData = ctx.createImageData(width, height); / * * at this point, We've got all the pixel information on the current canvas and with the width of the current canvas, For each image that is actually rendered (putImageData in the case of canvas), there is a buffer frame object (FrameBuffer). The FrameBuffer is used to store the data information of the image to be rendered next time. The FrameBuffer stores the information of the color array corresponding to imageData. data, the width and height information, the initial color (clear color) index the position in the color array by X and y. */ const buffer = new FrameBuffer(width, height); // Initial data buffer.setframeBuffer (imagedata.data); buffer.setClearColor(Color.BLACK); For (let x = 0; let x = 0; x < width; x++) { for (let y = 0; y < height; Const curposition = new Vec4(x + 0.5, y+ 0.5, 0, 1); const curposition = new Vec4(x + 0.5, y+ 0.5, 0, 1); Const isInner = isTriangleInner(curPosition, viewPosition1, viewPosition2, viewPosition3); // If the current point is inside the triangle, give the current coordinate pixel a Color isInner && buffer.setcolor (x, y, color.red); PutImageData (imageData, 0, 0); };Copy the code

At this point, we have drawn a simple triangle that still has many points and omissions to optimize and fill in below

Let’s look at how some of the abstracted algorithms are implemented, right

1. Viewport transformation

Let’s take a look at what the viewport transform does:

1. Move the coordinate origin from [-1, 1] * [-1, 1] [0, 0] origin to [0, 0] origin of [0, width] * [0, height]
2. Change the X and Y coordinates of the coordinate system from [-1, 1]^2 to width & height

To put it simply, a rectangle with width and length 2 is transformed into a rectangle with width & height by scaling and displacement. Through matrix representation, that is

So let’s do a simple implementation

/** The current function takes a coordinate represented by a vector and contains a 4*4 matrix represented above, multiplying the two, returns a new coordinate vector vector: a 1*4 matrix [x, y, z, w], w is used to assist the displacement operation, as shown in the matrix above: Function getViewPortPosition (vector) {// width = 0; // width = 0; // width = 0 canvas.width; height = canvas.height; const matrix = new Matrix( width / 2, 0, 0, width / 2, 0, -height / 2, 0, height / 2, 0, 0, 1, 0, 0, 0, 0, 1 ); Vec4MulMat4 (vector, matrix); vec4MulMat4(vector, matrix); }Copy the code

Vector * matrix example –> Returns a new vector

2. The cross product is used to determine whether the current coordinate point is inside the triangle

The cross product is calculated by the current coordinate (vector) and the three sides (vector) of the triangle respectively. If the symbol is the same, it is proved that it is on the same side with the three vectors, that is, the point is inside the triangle. Another more general judgment method will be given below, and the calculation method here is simple to understand the principle

3D model rendering

At this point we have a simple idea of how to render a basic triangle, but there is still a lot of confusion, such as

1. For a model with many triangles, it is obviously wasteful to have to completely iterate over the canvas every time you render a face.
2. If you look carefully, you will find that there is a very obvious jagged edge of the triangle. What is the cause? How to optimize
3. We can get the x and y coordinates of the vertices, and then look up the corresponding color in the texture image, and then the pixel coordinates in a face, if we determine their X and y coordinates
4. How to convert a 3D model into a 2D image that can be rendered, and how to deal with the perspective and occlusion relationship

Let’s work through each of these problems one by one, and then

1. Box-enveloping model: Delineate a smaller ergodic range by determining the maximum and minimum X and Y values of a triangle (or triangles)

const boundBox = getBoundBox(position1, position2, position3); for (let x = boundBox.minX; x < boundBox.maxX; x++) { for (let y = boundBox.minY; y < boundBox.maxY; y++) { .... }}Copy the code

2. Anti-aliasing: Combined with the actual scene, it is easy to understand the reason for aliasing, that is, the display of a single pixel is too large, or simply determining the display color of a pixel by a pixel cannot completely explain all the color information contained in the pixel.

When you set font Smoothing: None to an element, the word becomes very serrated

In practical application, there are many ways and means of anti-aliasing, suitable for different scenarios. Here is a simple anti-aliasing method for you to understand

Multi-sampling anti-aliasing

In the process of original drawing, when a path cover one pixel, by looking at the pixel center is in the path of the medial (graphics), to determine whether you need for the pixel shader, and at the edge of the two images intersect, a pixel need to take more responsibility, namely the transition between the two graphic information.

Therefore, it is impossible to complete the expression of such filtering situation only by relying on the center point coverage (true or false). According to the outline of thinking (increasing the sampling frequency), we divide four sampling points in a pixel, and judge that one (path) or graph covers several pixels. For example, if two pixels are covered, the transparency is 50%, and three pixels are 75%. This transition relationship can be expressed to a certain extent, making the edges of the graph look smoother

3. Interpolation: determine the data information of a point within three vertices by calculating the barycentric coordinates

Just as we identify a point in a line segment, we can express a point by the weight of three points, and by this way we can also determine whether the point lies in the plane composed of three vertices.

A (x1, y1) + b(x2, y2) = a(x1, y1) + b(x2, y2)Copy the code

4. Camera Introduction (View Transformation)

By introducing the camera, 3d coordinates (model data coordinates x, y, z) are converted to 2D coordinates (camera visible coordinates X, y) in the following three steps, as well as a viewport transformation described above

1. Model transformation: Simply put, it is to adjust the position of the model
2. Camera transform: Gets the relative position of the object observed from the camera Angle
3. Projection transformations (only perspective projections commonly used here): Map the 3D coordinates of the object’s vertices to a 2d plane [-1, 1]^2, and retain the z coordinates of its vertices for the calculation of occlusion relations (The Z coordinates of each pixel can be calculated by the barycentric coordinates of the three vertices)

Finally, through the viewport transformation method introduced above, expand the object from the coordinate system [-1, 1]^2 to [0, width] * [0, height] to complete the whole view transformation process

Camera transformation

Below is a code description of how a camera is created and manipulated

Class Camera {// Coordinates of the current Camera position, represented by a vector, e.g. (10, 10, 20) position; // Camera orientation, when the camera is applied, make the camera look at a certain coordinate point, to calculate the camera orientation vector direct; // The upward direction vector of the camera, determine the positive and negative up of the camera; LookAt (point) {.... : lookAt(point) {.... }}Copy the code

So let’s first of all, how do we calculate the vectors of the camera, the three coordinates of the camera that are perpendicular to each other

// By setting the camera position and the object position, This. Direct = point.x - this.position.x, point. Y - this.position.y, Point.z - this.position.z // Calculate and set the direct vector (at the same time unit the vector, Const unitLength = x*x + y*y + z*z + w*w this.x = x/math.sqrt (unitLength); . /** Calculate the X-axis vector by setting the temporary Y-axis auxiliary UP vector and cross-multiplying it with the direct vector. Then calculate the actual UP vector by X-axis vector and the direct vector. Up = vectorMultiply(XCoordinate, this.direct).normalize();Copy the code

So far, we have obtained the necessary camera position coordinates, object position coordinates, camera orientation vector and camera up vector for calculation

Next, we need to move the camera position & rotate it back to the origin of the world coordinates (that is, align the camera coordinates with the world coordinates), then we can get the relative position of the object observed by the camera (the object’s own coordinates are applied as the world coordinates)

X, 0, 1, 0, -this. Position. Y, 0, 0, 1, -this. Position.Copy the code

At this point, the camera’s coordinate origin is already aligned with the world’s coordinate origin, and only one rotation operation is required to make the two coordinates completely coincide

Because now we’re solving for the transformation matrix of rotation, which has the property of

The current matrix dot the transformation matrix is equal to the I matrix

Transformation matrix = inverse = transpose matrix (because: orthogonal matrix)

Therefore, only one transpose of the current coordinate is needed to obtain the corresponding transformation matrix

(
    XCoordinate.x, this.up.x, -this.direct.x, 0,
    XCoordinate.y, this.up.y, -this.direct.y, 0,
    XCoordinate.z, this.up.z, -this.direct.z, 0,
    0, 0, 0, 1,
).transpose();
Copy the code

So far, we have solved the direct relative position of the object and the camera, that is, the transformation matrix of the object. Now we need to map the transformed coordinate vertices to a coordinate system [-1, 1]^2 (Perspective projection transformation).

Perspective projection transformation

Because when the human eye observes objects, there is a relationship between near and far, and this ratio can be calculated

We only need to define a n value and z value as shown in the figure, and we can map the coordinate relationship by virtue of this ratio, and record the z point for use in depth testing (to judge the relationship between front and rear occlusion).

Finally, the corresponding matrix relationship is obtained as follows

If the image needs to be compressed back to the standard coordinate of [-1, 1]^2, the corresponding transformation matrix is shown as follows. In actual use, the X and Y coordinates of the corresponding viewport can be obtained by expanding it according to the viewport transformation introduced above

Depth test (Z buffer)

Stop here, we can get to a shade relations do not contain 2 d graphics, distance to the right graphic rendering, only the last step to handle the relationship between the shade, we known at this time, rendering graphics, is endowed with its corresponding color pixels, so in order to deal with shade relationship, for each pixel, we save a index value, Store the current z value and color information, if the following z value is greater than the current saved z value, then replace, that is, always save the maximum z value and color

Class ZBuffer {frameBuffer for storing color information, Constructor () {this.z = new Float32Array(canvas.width * canvas.height).fill(NaN); }}Copy the code

Finishing process

Function renderObj() {/** object 1 v -0.500000 -0.500000 0.500000 v 0.500000 -0.500000 0.500000 V -0.500000 0.500000 0.500000 VT 0.000000 0.000000 VT 1.000000 0.000000 VN 0.000000 0.000000 1.000000 vn 0.000000 0.000000 vn 0.000000 0.000000 1.000000 f 1/1/1 2/2/2 3/3/3 {face: [], // index v: [], // position n: [], // normal vector t: [], // texture coordinates x = vt. X y = 1-vt. Y} */ / loadModel(); model.setMapTexture(key, loadTextureImage()); Const obj = new ModelObject(model); Const camera = new camera (10, 10, 20).lookat (obj.position); Const zBuffer = new zBuffer (canvas.width, canvas.height); // Create zBuffer const zBuffer = new zBuffer (canvas.width, canvas.height); / / graphics rendering const mat = mat4MulLinked ([node. GetPositionMat4 (), camera, translte, camera. GetPerspectiveMat4 (), ]) for (let i = 0, len = model.face.length; i < len; Const face = model. Face [I]; Const texture = model.mat.get(face.key); // Calculate the bounding box model by vertex coordinates... const boundBox = getBoundBox(v1, v2, v3); For (let x = boundbox.minx; x < boundBox.maxX; x++) { for (let y = boundBox.minY; y < boundBox.maxY; Y ++) {// Calculate the barycentric coordinates const barycentric = getBarycentric(x+0.5, y+0.5, v1, v2, v3); / / is not in the triangle range tends not to calculate the if (barycentric. X < 0 | | barycentric. Y < 0 | | barycentric. Z < 0) {continue; } / / get the current coordinates to store the z value const z = getViewPositionZByBarycentric (v1, v2, v3 and barycentric); // If z is small, skip if (! zBuffer.depthTest(x, y, z)) { continue; } // zbuffer.set(x, y, z); Const u = getUByBarycentric(); const v = getVByBarycentric(); const color = texture.getColorByUV(u, v); // Set the corresponding color information buffer.setColor(x, y, color); PutImageData (imageData, 0, 0); }Copy the code