preface

OpenGL is a graphic application programming interface (API) of a cross-platform standard widely used and accepted in reality. It is huge and complex, and OpenGL ES is a subset of OpenGL after OpenGL removes all the redundancy. It takes handheld and embedded devices as the standard. Dominate today’s smartphones.

Note that at WWDC2018, apple announced that it was abandoning OpenGL, ES, and CL in favor of apple proprietary Metal.

Graphic rendering pipeline

OpenGL ES implements a graphics pipeline with programmable coloring, which consists of two specifications, OpenGL ES API and OpenGL ES Shader Language (GLSL, mainly used to write Shader code).

OpenGL ES 2.0 and OpenGL ES 3.0 are programmable graphics pipelines compared to the fixed feature pipelines of OpenGL ES 1.x series. Developers can write their own code for the vertex shader and fragment shader phases in the graphics pipeline. And OpenGL ES 3.0 is backward compatible with OpenGL ES 2.0. This means that applications written in 2.0 can continue to be used in 3.0.

In the real world, however, a large number of projects still use OpenGL ES 2.0, so this series of articles will try to cover both 2.0 and 3.0.

The translation of the Graphics rendering Pipeline is from Graphics Pipeline. It looks like the translation is ok, but the word “Pipeline” can be confusing or confusing. A graphics rendering pipeline is the “flow” of graphics rendering. Here’s a look at LearnOpenGL:

A Graphics Pipeline is the process by which a pile of raw Graphics data passes through a Pipeline, passing through various changes and finally appearing on the screen.

The graph below shows the OpelGL ES graphical pipeline:

The shaded box in the picture is the programmable stage of the pipeline and the most important part of our implementation of various effects and filters.

Vertex shader

The Vertex Shader implements a general programmable approach to Vertex manipulation.

The input and output of the vertex shader are shown above.

The input to the vertex shader is:

  • Vertex shader: A vertex shader program that describes operations performed on vertices
  • Vertex shader input(or attribute) /input(attribute) : Data for each vertex supplied with an array of vertices
  • Uniform variables /uniforms: Unspoiled data used by a vertex shader or fragment shader
  • Samplers /samplers: Special uniform variable types that represent vertex shaders using textures

The output of a vertex shader is called a vertex shader output variable. The mechanism that computes the vertex shader output value during pixel rasterization and passes it to the segment shader as input is called Interpolation for the vertex shader output assigned to each pixel vertex to generate each segment value

A new feature in OpenGL ES 3.0, Transform Feedback, allows the vertex shader output to be optionally written to an output buffer (in addition to, and possibly in place of, the fragment shader).

Let’s look at a vertex shader code:

// Vertex shader example
// Specify the OpenGL ES version
#version 300 es
// unify variables
uniform mat4 u_mvpMatrix;
/ / input
in vec4 a_position;
in vec4 a_color;
// Output, passed to the fragment shader
out vec4 v_color;

void main(a){
    v_color = a_color;
    gl_Position = u_mvpMatrix * a_position;
}
Copy the code

This program is very similar to the WRITING method of C language. It doesn’t matter if you don’t understand it at present. You can have a concept, which will be explained in detail in subsequent articles.

The input and output variables are modified with in and out keywords. Note that in, out, and INout keywords were added in 3.0 to replace attribute and varying. Gl_FragColor and gl_FragData are also removed.

In this program, a_position is the input vertex position attribute, a_color is the input vertex color attribute, and v_color is used to store the vertex shader output describing the color of each vertex. The internal variable gl_Position is declared automatically, and the shader must write the transformed position to this variable.

Primitive assembly

After the vertex shader, the next stage of OpenGL ES rendering the graphics pipeline is the Primitive Assembly. Primitives are geometric objects such as points, lines and triangles. Each vertex of the primitives is sent to a different copy of the vertex shader, and these vertices are combined into primitives during primitives assembly.

For each pixel, you must determine whether the pixel is within a visible space region on the screen, and if not, you may need to crop it. After clipping, the fixed point position is converted to screen coordinates. If the pixel is completely outside the visible region of space, it is discarded. You can also perform a flush operation, discarding them forward or backward based on the primitives. Once clipped and eliminated, the primions are ready to be passed on to the next stage of the pipeline, rasterization.

rasterizer

The rasterizer (Rasterization) is the process of converting primitives into a set of two-dimensional fragments that represent pixels that can be drawn on the screen. Corresponding primitives (points, lines, triangles) are drawn at this stage, and these fragments are then passed to the fragment shader for processing. The output of each fragment contains screen coordinates, colors, texture coordinates, and so on.

Fragment shader

The Fragment Shader implements a common programmable method for operations on fragments. Executes the fragment shader for each fragment generated during the rasterization phase.

The input to the fragment shader is:

  • Fragment shader: A fragment shader program that describes operations performed on a fragment
  • Input variable/INPUT (VARYING) : The rasterization unit outputs the vertex shaders generated by interpolation for each fragment
  • Uniform variables /uniforms: Uniform data used by a fragment shader or vertex shader
  • Samplers /samplers: A special unified variable type that represents the texture used by the fragment shader

The fragment shader can discard the fragment or generate one or more color values as output. The colors, depths, templates, and screen coordinates generated during the rasterization phase become inputs to the piece-by-fragment operation phase of the OpenGL ES graphics rendering pipeline.

Let’s look at the code for a fragment shader:

// Vertex shader example
// Specify the OpenGL ES version
#version 300 es
// Specify the precision qualifier
precision mediump float;
// Input vertex color, input from vertex shader output data
in vec4 v_color;
/ / output
out vec4 fragColor;
void main(a){
    fragColor = v_color;
}
Copy the code

The input to the fragment shader interpolates linearly between primitives and is then passed to the fragment shader. As you can see, the output of the vertex shader must be the same set of variables as the input of the fragment shader. FragColor will be the color passed to the next stage, and its value is set to the input color v_color.

Piece-by-fragment operation

The next stage of Fragment shaders is per-fragment Operations. Raster generated fragments of screen coordinates (x,y) can only modify pixels at position (x,y) in the frame buffer. In the fragment-by-fragment phase, some functionality and tests are performed on each fragment, which we won’t get into for now.

conclusion

This article introduces the graphics rendering pipeline of OpenGL ES, which can pave the way for our subsequent study. In addition, if you are still confused after reading this series, it’s ok to come back and have a clearer understanding of the whole process.