• CPU and GPU

    • CPU

      Central processing unit (CPU), is one of the main equipment of electronic computer, the core accessories in the computer. Its functions are mainly to interpret computer instructions and process data in computer software. CPU is the core component which is responsible for reading instructions, decoding instructions and executing instructions. The central processing unit mainly includes two parts, namely the controller, the arithmetic unit, which also includes the high speed buffer memory and the realization of the data between them, the control bus. The three core components of an electronic computer are the CPU, internal memory, and input/output devices. The main functions of the CPU are to process instructions, perform operations, control time, and process data

    • GPU

      Graphics Processing Unit (Graphics Processing Unit, abbreviated: GPU), also known as display core, visual processor, display chip, is a kind of specialized in personal computers, workstations, game consoles and some mobile devices (such as tablet computers, smart phones, etc.) to do Graphics and graphics-related computing work microprocessor

    • What is done during the rendering process?

      1. Determine which objects in the scene need to be rendered. An object will be rendered only if it conforms to certain rules. For example, objects that are within the camera’s visual cone can be rendered, while those that are outside the cone are Culled and not rendered
      2. The CPU collects data about each object to be rendered and assembles it into instructions called Draw Calls. A Draw Call contains a single mesh data and information about how it will be rendered. For example, which maps to use on the material. In some cases, object data with the same Settings will be merged into a Draw Call. The act of combining different object data into a Draw Call is called Batching.
      3. The CPU creates a packet called Batch for each Draw Call. However, batch contains data not only for the Draw Call, but also some data that is not relevant to the rendering performance issue we are talking about, so we will not expand it here. Once the batch work is complete, the CPU then does the following:
      4. Send a command to the GPU to change the Render State. This directive is called SetPass Call. The SetPass Call tells the GPU which Settings to use when rendering the next material. The SetPass Call is only issued by the CPU if there is a real need to change the render state from the previous material to the next material
      5. The CPU sends a Draw Call instruction to the GPU. Draw Call instructs the GPU to render the specific material according to the Settings defined by SetPass Call.
      6. In some cases, your batch may contain more than one render pass, depending on the pass block in your Shader code. A pass block requires the GPU to change the render state once. Therefore, for each additional pass in a batch, the CPU must send a new SetPass Call to the GPU, and then send a new Draw Call to the GPU.

      Meanwhile, the GPU performs the following tasks:

      1. The GPU completes rendering tasks according to the requirements of the CPU.
      2. If the current task is a SetPass Call, then the GPU changes the current Render State.
      3. If the current task is a Draw Call, the GPU renders the specified material. Render by vertex shader, slice shader

      To sum up, THE CPU calculates the model data to be rendered, and then sends instructions to call the GPU to render. First, the drawing model is read, and the CPU transfers the polygon read to the GPU, and the GPU draws the corresponding model skeleton according to the model data. Note that this step has no texture but only wireframe. The GPU puts the model data into memory. The GPU also materials and colors the model. The CPU gets the polygon information from video memory accordingly. The CPU then calculates the outline of the shadow generated by the light. After the CPU calculates, the graphics card does its job again, which is to fill the shadows with dark colors. No matter how big the graphics card is, the light and shadow are all calculated by the CPU, the GPU only has 2 jobs and 1 polygon generation. 2 for the polygon color.

  • Principles of computer rendering

    • Scanning way
      • Random scanning

        The older scanning method uses a random positioning method to control the operation of the electron beam (disadvantages can not display too complex images)
      • Raster scan

        Sweep up and down
    • Raster scan display system structure
      • Simplest raster graphics system structure
      • Commonly used raster graphics system structure
      • Advanced raster graphics system architecture

  • Screen tear and lag condition

    • Screen tearing


      Image rendering process:

      GPU render -> Frame Cache -> Video Controller -> Read Frame Cache information (bitmap) -> Digital to analog (digital signal -> model) ->(progressive scan) display

      It can be seen that the face-charging process is basically divided into two stages:

      1. The GPU stores the processed data (bitmap) into the frame buffer
      2. The video controller takes the data from the frame buffer and converts it to scan and display it so if the first step is too fast and the second step is too slow it will cause tearing. To summarize: First, the GPU saves the data to the frame buffer after processing. The perfect result is that the video controller just finishes displaying the data in the frame buffer and immediately updates the data of the next frame to the frame buffer. However, the processing speed of the video controller is a little slow. However, the GPU processing will soon show that the data of the third frame will be processed before the first frame is rendered and the data of the third frame will be replaced by the data of the second frame in the frame buffer, resulting in the loss of the second frame. After the first frame is completed, the third frame will be rushed to render, which causes the tearing phenomenon
    • In the case of a tear, apple Vsync + DoubleBuffering, Vsync means put a lock on the frame buffer, put it in the frame buffer, lock that buffer, and then unlock it when the video controller gets the data in the frame buffer and scans it to show that it’s done, This solves the tear problem, but the problem is that the rendering time of one frame is usually 16ms, and after 16ms, the next frame will be rendered. However, the CPU/GPU processing speed is greater than 16ms, so there is no data in the frame buffer when the next frame is rendered. The current frame is still displayed. (It is possible to drop frames without verticalsynchronization, but this is much better than with verticalsynchronization, because the GPU does not wait to overwrite the frame buffer after processing the data, thus ensuring that the CPU/GPU is still working during the rendering process.) In order to better solve this problem, three buffers were introduced. In fact, the larger use of GPU/CPU just can’t solve the root problem. To summarize: Frame loss is not a frame loss, but a waiting situation, which is actually the CPU/GPU processing speed can not keep up with the rendering speed, so far has not been able to fully solve the frame loss problem

  • Rendering flow under iOS

    1. Display logic:
      • CoreAnimation submits the session, including the layout state of itself and its subtree (View hierarchy);
      • RenderServer parses the submitted subtree state and generates the draw instruction
      • The GPU executes the drawing instruction
      • Display the rendered data
    2. Submit the multistage
      • Layout
        • Call the layoutSubviews method
        • Call the addSubview: method
      • Display
        • Draw a view with drawRect;
        • Draw string;
      • Ready to submit
        • Decode picture;
        • Image format conversion;
      • submit
        • Pack layers and send them to the rendering Server;
        • Recursively submit subtree layers;
        • If the subtree is too complex, it will consume too much and affect performance;

    To summarize: HandleEvents: Commit Transaction: Render Server (CPU) -> CoreAnimation -> Submit OpenGL -> GPU -> Render process (vertex data -> vertex shader -> Chip shader -> Runloop -> Display)