Bytedance Terminal Technology — Hooper
background
Data platform has a chart library based on graph syntax, ChartSpace, support web/ H5 /mini program, now received business requests to support the Flutter side.
To make it easier to understand, explain the concept of graph syntax a little bit, you can skip this paragraph if you already know it.
Graph grammar
Graph grammar (grammar of graphics) is through a set of grammar to describe any graphics, mainly from Wilkinson’s “The grammar of graphics”, can refer to The article: zhuanlan.zhihu.com/p/47550015
The main difference between graph syntax and normal chart syntax is that a graph syntax can produce a completely different graph by modifying the syntax description, whereas a normal chart needs to add a chart type. Graph syntax describes a near-infinite number of graphs, and a finite number of chart types.
For example: (snippets from segmentfault.com/a/119000004…
If we draw a bar graph based on graph syntax
When you change the coordinate system in the syntax to polar coordinates, it becomes a rose diagram
Adjusted coordinate metrics in syntax and added different colors to make a more complete rose diagram
Continue to tweak the syntax parameters to get the pie chart
In this example, if you use a generic chart (such as ECharts), you need at least four chart types, and the format of the chart data may differ. However, with graph syntax description, different graphs can be obtained by adjusting different syntax parameters.
Graph syntax adjusts syntax parameters to obtain different graphs, which provides more space for data expression. It belongs to a more professional graph engine, but it also brings more complex syntax rules.
Based on graph syntax, front-end (JS syntax) commonly used chart library:
- G2: Chart library of Ant Financial based on graph syntax, which is used through JS syntax
const ds = new DataSet(); const chart = new Chart({ ... }); . const dv2 = ds.createView().source(dv1.rows); Dv2. Transform ({type: 'regression', method: 'polynomial', Fields: ['year', 'death'], bandwidth: 0.1, as: ['year', 'death'] }); const view2 = chart.createView(); view2.axis(false); view2.data(dv2.rows); . chart.render();Copy the code- Vega: Open source graphical syntax framework that is used through JSON configuration
{
width : 500,
height : 200,
config : {
axis : {
grid : true,
gridColor : #dedede
}
},
...
}
Copy the code- ChartSpace: Bytedance charts library based on graph syntax, which is used through JSON configuration and has similar syntax to Vega
{
type : line ,
data : [],
labels : {
visible : false
},
axes : [
{
orient : left
},
{
orient : bottom
}
],
xField : x ,
yField : y
}
Copy the codeJson configuration syntax has better multi-endpoint consistency in cross-end, cross-language situations. The same JSON configuration is stored on the back end and the same graph can be drawn on multiple ends.
Graph library ChartSpace
ChartSpace is a chart library based on the graph syntax of the byte data platform. It already supports Web/H5 / Mini Program and now supports the flutter side.
The business expects multi-terminal collaboration, and the same data has consistent performance on different terminals. Take the line graph as an example:
plan
The conventional solution is to implement a set of graphics semantics for FLUTTER, parse the semantic configuration of Chartspace, and draw graphs of the same size. However, the development cost of this scheme is relatively high, so we choose another scheme: cross-end Canvas.
The principle is to render content drawn on the Web canvas used by Chartspace (js) to the Flutter canvas using cross-end technology.
This content cannot be displayed outside of the flying book document at this time
To achieve this solution, two problems need to be solved:
- Let me draw things
- String interactions together
Let me draw things
Core idea: Transfer canvas drawing instruction execution of chartspace(js) from JS to flutter execution. The goal is to align the flutter canvas with the Web canvas.
This is done by constructing a Mock Canvas object in JS, recording Canvas directives, and sending them to the Flutter side to be implemented by the Flutter Canvas.
The main work is to implement a Web Canvas API using Flutter Canvas.
This content cannot be displayed outside of the flying book document at this time
String interactions together
The input for user interaction is a touch event. Just convert the Flutter PointerEvent to a Web TouchEvent and enter it into chartspace.
Chartspace will then generate a new canvas directive and draw the new content in the Flutter canvas. The process is the same as the first rendering and the interaction is complete.
This content cannot be displayed outside of the flying book document at this time
The effect
The tooltip effect is the result of a finger click.
The benefits are: low cost implementation, low cost maintenance, and cross-end consistency.
Rendering performance comparison:
A lot of optimizations were made during development, with Graph rendering times going from 80ms to 50ms, and we’re still working on optimizations to get closer to the native experience. Later, our other partners will share their thoughts and practices on optimization.
| Across the Canvas | Pure Flutter | |
|---|---|---|
| The graph rendering | 52ms | 20ms |
| The tooltip rendering | 9ms | 0ms |
The data of cross-end Canvas starts from user input to the end of rendering graph, including bridge transmission and the time when chartspace (JS) generates Canvas instruction.
Pure Flutter is the execution time of the same canvas instruction into Flutter code.
We can see that the rendering performance differs somewhat from that of the pure Flutter mode, but it is also within acceptable limits, and it is difficult for the user to perceive the difference during normal chart interaction.
We believe that if the same chart is drawn by ourselves, it will have better performance. There is still room for optimization in the optimization of canvas instructions and the implementation of Web Canvas API.
Step on the pit & solution
In the process of practice, we have encountered many problems. Here are some representative ones to share
Canvas life cycle is different
The life cycle differences are as follows:
| Flutter Canvas | Web Canvas |
|---|---|
| Rendering does not save the canvas | Rendering saves the canvas |
| It’s redrawn every time | I’m going to build on what I did last time |
Our solution is to save the rendered result and continue drawing on the last rendered result
@override
void paint(Canvas canvas, Size size) {
final paintList = _repaint.consume();
ui.Picture picture = canvasRecorder.record(canvasId, size, _repaint.reverse, paintList);
if (picture != null) {
canvas.drawPicture(picture);
}
}
Copy the codeCanvas Context different
Context differs as follows:
| Flutter Canvas | Web Canvas |
|---|---|
| Saves a copy of the current transform and clip on the save stack. | Ctx. save Saved contents: |
| Each paint is a new canvas instance | After canvas is created, the instance does not change |
In response to the first problem, where the save/Restore content is inconsistent, we created a WebCanvas object to simulate a Canvas on the Web and manually manage the Save/Restore content
class WebCanvas { ... SaveStack saveStack = SaveStack(); SaveInfo get current => saveStack.current; . }Copy the codeFor the second problem, we create a CanvasRecorder object and hold a WebCanvas instance in it, consistent with the lifecycle of a Canvas instance on the Web
class CanvasRecorder { ... CanvasHistory getCanvasHistory(String canvasId) { if (! hisMap.containsKey(canvasId)) { hisMap.putIfAbsent(canvasId, () => CanvasHistory(canvasId)); } return hisMap[canvasId]; }... } class CanvasHistory { ... final ChartSpaceCanvas chartSpaceCanvas = ChartSpaceCanvas(); . } class ChartSpaceCanvas { ... final WebCanvas webcanvas = WebCanvas(); . }Copy the codeCanvas defaults to different values
There are many differences in the default values of Canvas. We directly set the default values according to the standard of Web Canvas without careful statistics. Roughly speaking, there are differences in the following attributes:
- transform
- fillStyle
- strokeStyle
- strokeMiterLimit
- font
Take Transform as an example, transform actually maintains a 4 * 4 transformation matrix (DOMMatrix object), and setTransform method on the Web sets values at different positions of the transformation matrix
The transformation matrix is directly manipulated on Flutter
However, the default values of the transform matrices of the Web Canvas and the Flutter Canvas are inconsistent
| Flutter Canvas | Web Canvas |
|---|---|
| 00 0 00 00 00 00 00 00 00 0 | 1 00 00 1 0 00 0 1 00 00 1 |
So here’s the solution:
class Matrix4Builder { static Matrix4 webDefault() { final matrix4 = Matrix4.zero(); Matrix4. SetEntry (0, 0, 1.0); Matrix4. SetEntry (1, 1, 1.0); Matrix4. SetEntry (2, 2, 1.0); Matrix4. SetEntry (3, 3, 1.0); return matrix4; }}Copy the codeThe Bridge needs the synchronization API
The Mock CanvasRenderdingContext object is used to record the Canvas directive, but there are many methods on the CanvasRenderdingContext object that need to synchronize the API, such as measureText.
But regular Bridge communication is
This content cannot be displayed outside of the flying book document at this time
The Flutter communicates asynchronously with iOS/Android, so FFI communicates directly with the JS Runtime to ensure synchronization
This content cannot be displayed outside of the flying book document at this time
Intercept part of the code implementation:
Pointer<Utf8> funcMeasureTextCString = Utf8.toUtf8('measureText');
var measureTextFunctionObject = jSObjectMakeFunctionWithCallback(
_globalContext,
jSStringCreateWithUTF8CString(funcMeasureTextCString),
Pointer.fromFunction(measureTextFunction));
jSObjectSetProperty(
_globalContext,
_globalObject,
jSStringCreateWithUTF8CString(funcMeasureTextCString),
measureTextFunctionObject,
jsObject.JSPropertyAttributes.kJSPropertyAttributeNone,
nullptr);
free(funcMeasureTextCString);
Copy the codeSummary & Prospect
To sum up, we implemented Flutter ChartSpace at a low cost using a cross-canvas approach and achieved good performance in practice.
This is also due to the reasonable architecture design of ChartSpace itself, which can effectively shield the differences of different platforms and languages by defining the semantics of graphics through JSON configuration.
Because ChartSpace is an implementation based on graph semantics, it requires more computation than custom chart types, which affects rendering performance. But now it also supports step rendering, in big data and complex graphics, can gradually render complete graphics with progressive effects, without compromising the user experience.
Flutter ChartSpace does not support stepwise rendering yet. There is still a lot of room for improvement in the current design, which we will continue to explore.
In the future, we will consider further expansion in two directions:
Design apis that are easier to use
While graphics syntax is powerful, it also brings complexity. We can wrap a layer of API around graphics syntax to cut down the cost of using common graphics.
Ant Group’s g2plot, for example, is a encapsulation based on G2, providing a more concise syntax and quoting a description from g2plot
The description from: zhuanlan.zhihu.com/p/339275513
const line = new Line('container', {
data,
xField: 'year',
yField: 'value',
});
line.render();
Copy the codeYou can compare the syntax of g2plot with the syntax of G2, which is shown in the graph syntax section of this article.
Expand more end/technology stacks
In practice, we found that the same technology can be extended to more stacks, such as iOS/Android/RN
This content cannot be displayed outside of the flying book document at this time
Open source
ChartSpace and Flutter ChartSpace are products inside bytes. ChartSpace has worked on a wide range of data products and other businesses, tested in different scenarios, including tiktok data analytics, and now has open source plans. Flutter ChartSpace also needs to polish its internal scene before considering open source.
Flutter ChartSpace will open source after ChartSpace, expected later this year.
🔥 Volcano Engine APMPlus application Performance Monitoring is a performance monitoring product for volcano Engine application development suite MARS. Through advanced data collection and monitoring technologies, we provide enterprises with full-link application performance monitoring services, helping enterprises improve the efficiency of troubleshooting and solving abnormal problems. At present, we specially launch “APMPlus Application Performance Monitoring Enterprise Support Action” for small and medium-sized enterprises, providing free application performance monitoring resource pack for small and medium-sized enterprises. Apply now for a chance to receive a 60-day free performance monitoring service with up to 60 million events.
👉 Click here to apply now