Off-screen rendering with normal rendering

There are two loading processes for the data ultimately displayed on the screen

  • Normal render loading process

  • Off-screen rendering loading process

The difference between them is that the off-screen rendering has an extra off-screen buffer. What does this buffer do? Let’s talk about it more carefully

First, the normal rendering process

Normal rendering flow

After CPU calculation and GPU rendering, the data in APP will be stored in the frame buffer, which will be taken out from the frame buffer by the video controller and displayed on the screen.

  • In the rendering process of GPU, images displayed on the screen are stored in the frame buffer according to the big painter algorithm in the sequence from far to near

  • After the video controller reads a frame from the frame buffer and displays it on the screen, it will immediately discard the frame without any reservation. The purpose of doing this is to save space and display each frame separately on the screen without affecting each other.

Off-screen rendering process

When the App needs to perform additional rendering and merging, such as setting rounded corners for buttons, we need to round corners + crop all layers in the UIButton control, and then store the merged result in the frame cache, and then take it out from the frame cache for display on the screen. In this case, in the normal rendering process, We can’t round all the layers because it’s one layer and one layer at a time. Therefore, we need to put the processed results into the off-screen buffer in advance, and finally combine several layers and store them in the station buffer. Finally, the effect we want to achieve is on the screen.

In plain English, the off-screen cache is a temporary buffer for data that is not currently in use but will be used for future operations.

  • Off-screen rendering brings us convenience as well as serious performance issues. Since the off-screen Buffer in off-screen rendering is an extra storage space, it also takes time to transfer data to the Frame Buffer, so it is still possible to drop frames during the transfer process.
  • The off-screen buffer space is not infinite, it is capped, can only be 2.5 times the size of the screen

So why do we use off-screen rendering when we know there are performance issues?

  • It can handle some special effects that cannot be done in one go and require the use of an off-screen buffer to save the intermediate state, which has to be rendered off-screen. In this case, the off-screen rendering is automatically triggered by the system, such as rounded corners, shadows, Gaussian blur, rasterization, etc., which are often used
  • It can improve the efficiency of rendering. If an effect is realized several times, it can be rendered in advance and saved to the off-screen buffer for reuse. This situation needs to be triggered manually by the developer.

Another reason for off-screen rendering: rasterization

When the value of this property is YES, the layer is rendered as a bitmap in its local coordinate space and then composited to the destination with any other content.

When rasterization is enabled, the layer will be rendered into bitmaps and stored in the cache, so that the next use can be directly reused to improve efficiency. For the use of rasterization, there are several suggestions:

  • If layer cannot be reused, there is no need to turn on rasterization
  • If the layer is not static and needs to be modified frequently (such as during animation), turning on rasterization can actually affect efficiency
  • Off-screen rendering cache content has a time limit. If it is not used within 100ms, it is discarded and cannot be reused
  • Off-screen rendering has limited cache space, which is 2.5 times the size of the screen. Exceeding 2.5 times the size of the screen pixels will also fail and cannot be reused

Trigger time for off-screen rendering in rounded corners

Before talking about rounded corners, first explain the composition of CALayer, as shown in the figure, it is composed of backgroundColor, contents, borderwide & Bordercolor. Related to the fillet trigger off-screen rendering that we will explain.

Rounded corner setting does not take effect problem!

When writing normal code, such as UIButton setting the rounded corner, when setting the button image, cornerRadius, borderWidth, borderColor and other properties, we found that it did not achieve the desired effect

let btn0 = UIButton(type: .custom) btn0.frame = CGRect(x: 100, y: 60, width: 100, height: 100) // Set btn0.layer.cornerRadius = 50 // Set border width and color btn0.layer.borderWidth = 2 btn0.layer.borderColor = Uicolor.red. CgColor self.view.addSubView (btn0) // Set background image btn0.setimage (UIImage(named: "mouse"), for:.normal)Copy the code

This is the effect, you can see, we set the button picture is still square

Ounds must be set to true to get the desired effect. MasksToBounds must be set to true to get the desired effect. The solution is simple, but the principle is not carefully studied by most people.

Here’s what apple’s official documentation says about rounded corners:

The official document tells us that setting the cornerRadius will only set the rounded corners for backgroundColor and boder in CALayer. It will not set the rounded corners for contents. If contents needs to be rounded, You need to set both maskToBounds/clipsToBounds to true.

So we can understand that the root reason why rounded corners don’t work is because you don’t have rounded corners for contents, and the image that the button sets is inside of contents, so what you see on the screen is that the image didn’t have rounded corners clipped.

So I’m going to do a couple of pieces of code to tell you when an off screen render is triggered in the rounded corner Settings first of all, you need to turn on the off screen render color flag in the emulator, okay

1. Only set the background color +border button

let btn01 = UIButton(type: .custom) btn01.frame = CGRect(x: 100, y: 200, width: 100, height: 100) // Set the rounded corner btn01.layer.cornerradius = 50 // Set the border width and color btn01.layer.borderWidth = 4 btn01.layer.borderColor = CgColor self.view.addSubview(btn01) // set the backgroundColor btn01.backgroundcolor = uicolor.greenCopy the code

In this case, either using the default maskToBounds/clipsToBounds (false) or setting it to true will not trigger the off-screen rendering. The root cause is that there are no layers in contents that need to be rounded.

Case 2: Button setting background image +boder

let btn0 = UIButton(type: .custom) btn0.frame = CGRect(x: 100, y: 60, width: 100, height: 100) // Set btn0.layer.cornerRadius = 50 // Set border width and color btn0.layer.borderWidth = 2 btn0.layer.borderColor = Uicolor.red. CgColor self.view.addSubView (btn0) // Set background image btn0.setimage (UIImage(named: "mouse"), for:.normal)Copy the code

  • If you can use the default maskToBounds/clipsToBounds (false), this is the case where rounded corners do not work

  • MaskToBounds/clipsToBounds changed to true

As can be seen from the screen display, off-screen rendering is triggered at this time, because the rounded corner setting needs to be clipped to all layers, while maskToBounds clipping is applied to all layers. From a normal rendering point of view, layers are disposable. Now, our rounded corner setting requires the combination of 3 layers, so the layer processed first is saved in the off-screen buffer, and when the last layer is processed, the combination is rounded and clipped, so the off-screen rendering will be triggered

Btn.layer. cornerRadius = 50&& btn.clipstobounds = YES Will automatically trigger off-screen rendering? The first step is to turn on the off-screen rendering detection, and turn on color offscreen-Rendered in the simulator. Once turned on, those layers that need to be rendered off-screen will be highlighted yellow, which means the yellow layers may have performance issues.

Let’s write the following code:

Note: On my computer, the emulator screen that appears when I select model 11-Pro-Max is:

If you’re careful, you’ll notice that 1 and 3 have turned yellow, and you can’t see it here, when I chose the iphone8:

It is obvious here that 1 and 3 have turned yellow, marked as triggering off-screen rendering. Personally, I think this is a bug in the emulator, if your computer does not have this problem, please ignore it, if it does, try to choose another model!!

First of all, let’s popularize the hierarchy of CALayer :CALayer is composed of backgroundColor, content contents, and edge border wide &border color

** The cornerRadius documentation makes it clear that the cornerRadius setting only works for CALayer backgroundColor and borderWide & BorderColor. If contents has content or the background of the content is not transparent, this only works if you can set masksToBounds to true, where the two properties are combined to produce an off-screen render. This explains why codes 1 and 3 trigger off-screen rendering, but 2 and 4 do not trigger off-screen rendering

Solutions:

(1) The background draws the rounded corner picture, and the foreground sets it

(2) For the layer with no content or content with transparent background (no area other than rounded corner), directly set the layer backgroundColor and cornerRadius properties to draw rounded corner.

(3) Using the blending layer, overlay the translucent layer of the corresponding mask shape on top of the layer

sublayer.contents=(id)[UIImage imageNamed:@”xxx”].CGImage;

[view.layer addSublayer:sublayer];

(4)- (UIImage *)yy_imageByRoundCornerRadius:(CGFloat)radius corners:(UIRectCorner)corners borderWidth:(CGFloat)borderWidth borderColor:(UIColor *)borderColor BorderLineJoin :(CGLineJoin)borderLineJoin this method is YY_image processing method of rounded corners, you can go to download YY_image view source code

Other situations trigger off-screen rendering and solutions:

——> Using a blend layer, overlay a translucent layer on top of the layer with the corresponding mask shape

2. EdgeAntialiasing —–> Do not set allowsEdgeAntialiasing attribute to YES(default: NO)

AllowsGroupOpacity (the group is opaque. After CALayer’s allowsGroupOpacity property is enabled, the maximum visual transparency of the child layer is the same as that of its parent layer (corresponding to the alpha of UIView). This feature has been turned on globally by default since iOS 7, in order to keep the child view as transparent as its container view. ——-> Turn off the allowsGroupOpacity property and control layer transparency as required by the product

4. We (shadow) — — — — — – > Settings after the shadow, set CALayer shadowPath, the layer. The shadowPath = [UIBezierPath pathWithCGRect: the bounds]. CGPath;

CALayer’s ultimate solution for off-screen rendering: shouldRasterize to YES when the content of the view is static. This is the most practical and convenient solution. view.layer.shouldRasterize = true; view.layer.rasterizationScale = view.layer.contentsScale;

ShouldRasterize:

1. There is no need to open layer if it is not needed

2. If the layer is not static and needs to be changed frequently, such as in animation, turning on rasterizer can affect performance

3. The off-screen rendering cache has a time limit. When it exceeds 100ms, the content that is not used will be discarded and cannot be reused

4. Off-screen rendering cache has space limit, exceeding 2.5 pixels of screen will be invalid and unusable

Special note: When the view content is dynamic change (such as background after downloading pictures to switch to the main thread Settings), using this scheme will increase the system load.

Conclusion:

(1) Off-screen rendering is triggered by the system, and then there is an off-screen buffer. There is no causal relationship between the off-screen buffer and the secondary buffer mechanism of frame buffer mentioned in my last article

Btn.layer. cornerRadius = 50&& btn.clipstobounds = YES The cornerRadius documentation clearly states that the cornerRadius setting only applies to CALayer backgroundColor and borderWide & BorderColor, if the contents have content or if the background of the content is not transparent, This works only if masksToBounds is set to true, when the two properties are combined to produce an off-screen render

(3) When uitableVIewcell triggers off-screen rendering, it will open up the off-screen buffer frequently during sliding, which will cause the tanleView sliding to stall. If the view content is static, shouldRasterize(rasterize) to YES, this solution is the most practical and convenient. However, when the view content is dynamically changing (such as background image download after switching to the main thread Settings), using this scheme will increase the system load.

(4) We now have three choices: on-screen rendering, off-screen rendering, and CPU rendering. Which one should we use? This depends on the specific usage scenario. · Try to use the current screen for rendering. In general, we try to use the current screen for rendering in view of the performance problems that off-screen rendering and CPU rendering may bring. Off-screen rendering VS CPU rendering

conclusion

  • When only backgroundColor, border is set, and contents does not have subviews, no mattermaskToBounds / clipsToBoundsistrueorfalseWill not trigger an off-screen render
  • Set when contents has subviewscornerRadius+maskToBounds / clipsToBounds, will trigger an off-screen render, but that doesn’t work in UIImageView, where only images + is setmaskToBounds / clipsToBoundsDoes not trigger off-screen rendering,Apple optimized UIImageView and I think it's just drawing image directly on top of contents so you don't have to set the background color and you only have to render a layer, so you don't have to use the off-screen buffer,Thanks a lot heredas112So it does not generate an off-screen render. If a background color is added, it will trigger an off-screen render.

So, in summary, whether an off-screen rendering is triggered depends on whether we need to use an off-screen buffer