preface

This week, I was lucky to attend the whole process of QCon 2021 Shanghai station (2021/10/21-2021/10/23). I mainly listened to AIOps, performance optimization, future digital world, new trends of mobile, new trends of artificial intelligence and three-dimensional security defense system of Internet enterprises. Overall, microservices and AI are very hot topics, the presence of the big front end is low (manual dog head), and as an iOS developer with a background in end-to-end algorithmic engineering infrastructure and a wild security game, it feels a bit dangerous.

I used to write articles biased toward the bottom of the technology, write up more handy, today suddenly write this kind of partial narrative nature of the article feel a little mengbi. Now I will make a summary of this experience, hoping to help you.

trip

My colleagues and I arrived in Shanghai on the evening of October 20th, and then checked into the Hotel Shanghai Baohua Marriott Hotel held by QCon by subway and on foot. The hotel was of high grade and very luxurious:

Then came the morning sign-in and keynote speech on 21st. On the morning of the first day, the venue was a large banquet hall composed of four banquet halls, which gave me a sense of immersion at the conference site (I had never been to the conference site, manual dog head) :

Since 21 afternoon, until 23 evening, QCon is the organization form of multiple at the venue, I for all kinds of speech according to their interests and job relevance sorting, hardly a fall of repeated horizontal jump between the multiple at the venue, summing-up meeting about conference contents listed in the next chapter of this article.

It was fun to attend on weekdays, and the meetings ended around 6pm every night, so I had plenty of time to hang out and dine with friends in Shanghai. During our graduate students have a “gourd brothers” organization, seven people, including me all different technical direction, often late at night with technologies such as activity, since work everybody dispersed in Beijing, hangzhou and Shanghai, with families of their own after it is difficult to gather together, as a student, this happen to have an opportunity and a junior partner in Shanghai eat two meal, A meal in the evening, very well-attended; The other meal was a midnight snack that spanned the wee hours of the morning. The participants included only two “single students in Shanghai” except me (manual dog head) :

Compared to Hangzhou, Shanghai’s nightlife is more abundant, and the subway is still full of people after 10 PM:

The night scenery of the Bund is very beautiful. There are even old men and women singing open-air KTV and dancing in the square opposite the Bund.

On the whole, I enjoyed the meeting very much, not only learning knowledge, but also seeing beautiful scenery and spending time with friends I haven’t seen for a long time.

The meeting summary

In general, there are few topics that exactly match the professional direction of your work. Therefore, most of the summaries here may not be so systematic, but more scattered points, hoping to be helpful to you.

Toward Software Performance Evaluation at Scale: A Journey

This is the first keynote speech on the morning of the first day of the conference. It is the keynote speech of large-scale software performance evaluation brought by Principal Engineer Kingsum Chow from Ali. It mainly introduces the evaluation method of performance change caused by software and hardware upgrade and configuration change. Here we introduce a performance quantification indicator called RUE, RUE = Resource Usage/Work Done. To put it simply, the lower the ratio, the less resources consumed to complete the limited task, namely, the higher the performance.

At this point, it seems that the keynote is more about operations than clients, but Kingsum Chow’s Simpson’s Paradox, introduced by an example, should inspire everyone:

A trend appears in several different groups of data but disappears or reverses when these groups are combined.

In simple terms, even if in each set of group assessment indexes are down, the group after the merger may get the overall index increased after the calculation of the opposite conclusion, Kingsum here to performance optimization example of a contains three group as an example, from the chart we can see that the performance of the three groups decreased, but the overall performance is slightly increase:

It can be seen that the Speedup calculated by App Group 1-3 combination is greater than 1, that is, the overall performance improves slightly, but the Speedup of all groups is less than 1, that is to say, the performance of all groups decreases, which is a very counterintuitive mean trap. Then Kingsum used vector method to simply prove this paradox:

Here, the blue vector represents before optimization, and the green vector represents after optimization. The horizontal axis can be understood as load, while the vertical axis represents resource usage. Therefore, the smaller the slope is, the less resources are used with the increase of load, namely, the better the performance. But if we put all the green and blue together to get the vector which are a vector may be found below the green column appears in the blue column, namely the illusion of the overall performance improvement, this may be due to contain the animation in the final without public show in the PPT, here I attached a picture of a scene for your reference:

This paradox tells us that when designing AB experiment or other data evaluation indexes, we must deeply understand the meaning of evaluation indexes, and it is easy to get completely opposite conclusions if we infer only from surface data.

Mission Critical AI’s road to commercialization — autonomous driving samples

This keynote speech was brought by Wu Gansha, co-founder, chairman and CEO of Yu Shi Technology, and mainly introduced the commercial road of autonomous driving. Since this PPT is not open to the public and I am an outsider, here is a summary of some points learned from the speech:

1. This speech is very wonderful. The speaker highlights the key points through the cadence and intonation. The overall structure of PPT is perfectly integrated, and there is almost no feeling of boredom or fault in the middle.
2. The RAND Corporation estimates that 11 billion miles will be needed before they can make safety judgments with reliable data. That became a big hurdle for commercial self-driving, but Tesla used shadow mode to quickly gather data on 11 billion miles that would have taken hundreds of years by selling hundreds of thousands of cars and running L4 while humans were at the wheel. This idea of population divide and conquer is also present in algorithms like reCAPTCHA;
3. Reliability is safe in the known range, and robustness is also safe for unknown boundaries.
4. Metaverse may provide samples and training grounds for autonomous driving.

Java multi-platform performance optimization under multi-core Architecture of One Cloud

This lecture was brought by Wang Zhuo, intelligent JVM architect of Alibaba/Ali Cloud, and mainly introduced the underlying performance optimization methods of Java on AARCH64 and RISC-V. There was no public PPT for this lecture, and my summary is as follows:

• Here, some parallel computing scenarios are taken as examples to introduce the means of improving vectorization through SIMD. In view of Java’s insufficient freedom in using SIMD instructions, Vector API, the open source Java Vector optimization feature of Alibaba Dragonwell, is introduced. We then demonstrated the use of the Vector API with simple examples such as Euclidean distance and cosine similarity calculation.

SenseMARS Mars Mixed reality platform and application development implementation and challenges

This speech was brought by Fuxiuyuan, technical director of Sensetime/Mobile Intelligent Business Group. It mainly introduced the immersive virtual-real fusion visual effects and interactive experience realized through 3D reconstruction, spatial recognition and positioning technologies, including indoor and outdoor navigation, interactive games, AR bus and other scenes. Based on this landing of SenseMARS Mars Hybrid reality platform, several key points mentioned here include:

1. Data collection can be carried out through the scheme of head-mounted panoramic camera + mobile phone App auxiliary information collection collection. Users and businesses can complete data collection independently according to the rules of S-shaped path and path closed-loop.
2. It provides cross-platform support and can even run directly in H5 and small programs. It is mentioned here that aiming at the problem that SLAM cannot run under JS technology stack, the existing algorithm is transplanted with WebAssembly to improve the performance by 20 times.
3. As visual positioning and path planning are related to specific scenes and many problems need to be reproduced on site, ARCore provides Recording and Playback API to record data tracks and add business-related data to support Playback, facilitating remote analysis and problem solving.
4. The cumulative errors of SLAM are corrected by keeping the local SLAM coordinates and cloud coordinates through silent localization. Solve similar scenes through bluetooth, GPS, geomagnetic and other auxiliary signals.

Technical challenges behind byteDance’s Intelligent service implementation on ByteNN

This talk was brought by Xavier, bytedance/Intelligent Creation ByteNN project leader. It mainly introduced the end-to-end application of ByteNN, the bytedance inference engine, and the construction of supporting algorithm engineering infrastructure. My summary is as follows:

1. ByteNN model supports dynamic delivery based on configuration at the end and defines the threshold of executable algorithm by means of model scoring to avoid any impact on user experience.
2. ByteNN provides a tool chain for model optimization, automated rack testing, anomaly detection and reporting to assist the algorithm from development to implementation.
3. End-to-end algorithm pre and post processing, model call and other codes need to be implemented on the end, which inevitably requires the intervention of the client. According to the content of the speech, byte provides the end-to-end Python execution link and an automatic code conversion tool from Python to C++. And students make algorithms such as template code generation to a certain extent, can independently accomplish algorithm on the client side validation and deployment, but in our experience this is not realistic, because the algorithm classmates don’t understand end side, most people do not have the corresponding engineering capability, the algorithm validation and deployment link need the engineering team to engage in heavy. In the final question-asking session, I also asked the lecturer about the cooperation between the algorithm team and the engineering team. Generally speaking, ByteNN’s end-to-end algorithm engineering infrastructure is good, but there is still a certain distance for the algorithm to independently complete training, verification, deployment, feedback and redevelopment of the link. At present, our mobile terminal intelligent team is also building and improving the end-to-end algorithm engineering infrastructure through a variety of means, and has basically realized the independent verification and deployment of the algorithm. For details, please refer to our article published on Alibaba mobile technology mp.weixin.qq.com/s/f_YI_gwbz… And welcome to join us!

3d modeling technology towards the age of the meta-universe

This lecture was brought by You Wen, the 3D ARTIFICIAL intelligence director of Alibaba/Taoshi Technology, and mainly introduced the Object Drawer, a high-performance 3D reconstruction and rendering solution based on NeRF neural rendering technology. Its working principle is to train a neural network with images and camera pose based on image restoration as inputs. Different from ordinary neural networks extracting information from inputs, the spatial information of NeRF neural network is actually stored in the neural network. The process of modeling is the process of training, and the process of rendering is the process of reasoning.

Compared with traditional NeRF network, Object Drawer can solve problems such as long training time, large model size, slow inference speed (low frame rate), poor visual Angle robustness, and high-frequency texture distortion. Achieve 1080p, complete training within 4h, 30 FPS on mobile phone, high degree of texture restoration, model volume of only 20M and other indicators, support shoes, bags, handwork, blocks, porcelain POTS, hats, sofas, coffee tables, cabinets and other categories, allowing to complete shooting in general scenes without professional equipment. It has been gradually applied in Taobao App.

In addition to the 3d reconstruction technology based on neural network, Apple also released a 3D reconstruction scheme based on the traditional Photogrammetry algorithm along with macOS Monterey this year. By inputting pictures and a series of parameters containing camera parameters, the algorithm will produce objects containing maps. Trained objects can be easily added to ARKit scenarios through Apple’s Reality series of tools.

3D reconstruction technology provides us with the possibility of mapping objects from the real world to the virtual world. We believe that the popularization of these technologies will bring new possibilities for AR/VR, games and other industries.

Ctrip SDL fine construction road

This speech was brought by Tu Hongwei, the principal of basic security of Ctrip Information Security Department. He mainly introduced business threat modeling, application of large-scale IAST technology, security left shift scheme based on SAST/SCA + GitLab Pipeline and vulnerability management strategy. As I am also involved in some client-side analysis work, I pay close attention to IAST and security aspects, so the following summary will be more focused on this part.

At present, many enterprise servers and clients have the practice of deploying similar passive detection technology, among which the server is mainly based on IAST passive piling, which can detect and intercept SSRF and common injection vulnerabilities. The following is the IAST solution of Ctrip:

When a server is attacked by SSRF or injection, the IAST detects exceptions based on the rules and generates alarms and blocks them. This ensures continuous and dynamic application security. But IAST is not a panacea, for example against “legitimate” inputs caused by logic bugs.

In addition to the server IAST, Ant introduced the security section defense system on the 2020 BCS. Its core idea is to establish a set of security layers that are interwoven and parallel with the business, so that the security can penetrate into the business logic and realize careful observation and attack and defense. From the level of the whole train of thought and server-side IAST consistent (below come from the ants about safety aspects made public developer.aliyun.com/article/774…

Since the release process of the client is long, especially the iOS client needs to be reviewed by the App Store, the deployment may require T+1 for ordinary customers, which is doomed to the insufficient response speed of the client to 0Day, and it is easy to be taken by surprise once the vulnerability is exploited, and the harm of the client vulnerability is often great. For example, a simple scan or a passive redirect can steal your account. In the past, we could only resort to Hotpatch for temporary repair and then add an emergency release version. However, HotPatch is not a panacea, and it is difficult to carry out fine-grained repair or repair at all in many cases. In this case, the value of the safety section is reflected. The rules-based security slice actually provides some hotpatch capabilities, in addition to the ability to detect and intercept potential problems in advance based on cloud analysis, which has great value for client security but increases the risk of white hats being known internally (manual dog heads).

Codeql-based small program privacy detection and risk management

This speech was brought by Jin Qianren, senior security engineer of Alipay Technology Department, and mainly introduced some applications and implementation of codeQL-based static analysis technology in Alipay small program code risk detection. CodeQL is an automated code analysis tool based on AST. It generates code database through access to source code compilation process, and then can query variable definition, function call and even data flow based on SQL-like rules. This year, the author also pays attention to some utilization of CodeQL in vulnerability mining. For example based on CodeQL o0xmuhe teacher wrote XNU source level up (o0xmuhe. Making. IO / 2021/02/15 /… .

The speech brought by Alipay students focused on some detection of CodeQL’s illegal use of privacy API, including the first screen popup, direct disclosure of sensitive information and repeated compulsory authorization. The basic idea was to mark all entry functions as source and all authorization functions as sink based on stain analysis. Tracing back from sink to source through control flow query and auxiliary data flow analysis provided by CodeQL, for example, abnormal logic detection of repeated pop-ups after authorization rejection is given below:

Through CodeQL, on the one hand, various program analysis rules can be established to reduce business security risks, on the other hand, it can also be used to open source code base, software and system vulnerability mining, through this standardized way can greatly improve work efficiency.

Break through the last kilometer of data chain protection: Tencent Confidential Computing Application Practice & Secure Computing at your fingertips

These are the last two topics of QCon Shanghai 2021 about the three-dimensional security defense system of Internet enterprises, respectively given by Tencent Blade Team technical director Zhang Bo and Ant Group senior technical expert Gu Zongmin (xianghe), mainly introduced the principle and application scenarios of secret computing. We know that the data security risk mainly in storage, transmission and use of the three stages, including storage and transmission have relatively mature technologies to protect, and use is often to decrypt the data directly to memory, malicious software by taking a memory access control flow or way to steal data, confidential calculation mainly for data usage scenarios, The logic of data computation is secured through special units provided by various hardware (e.g. TEE, TrustZone, SGX, etc.).

TEE, for example, in order to protect the security of the user’s data, the data submitted to the server computing, the first symmetric encryption key generated by the user for data encryption, and then through the remote proof mechanism is the key to the TEE, the TEE to receive the user’s encrypted data, then finish decryption, in internal operation and then encrypted to ensure safety.

An important problem here is that TEE is different from conventional operating system, resulting in high cost of software adaptation, so how to make software run in a trusted execution environment without perception has become an important topic, then LibOS came into being, the so-called LibOS, That is, Lib(userland) + OS(syscall), that is, LibOS acts as a bridge between userspace <-> kernel <-> TEE, enabling applications to run unmodified in trusted execution environments such as TEE.

In the end, the lecturer introduced ant Group’s open source TEE OS solution, Occlum, which was developed based on Rust and Intel SGX SDK. By default, it uses transparent encryption file system, can use TLS network encryption communication protocol based on remote proof, and has high-performance multi-process solution. Support for multiple processes to share a Enclave, currently has implemented 200+ mainstream Syscall, support Redis, Tensorflow and Pytorch and other mainstream software directly running on it.

conclusion

In general, it was a fruitful trip, which not only allowed me to have a deeper understanding of the latest research directions and achievements in the industry, but also made me meet many good friends. Unfortunately, there are few issues related to the client, and maybe the client is really coming to an end (manual dog head).

At the end of the article, another advertisement for our team:

Welcome to join alibaba Taoshi Technology Department-Duan Intelligence team, responsible for building the industry-leading open source reasoning engine MNN and one-stop machine learning software MNN workbench. In Ali, we are responsible for AR platform of core e-commerce and new forms of commodity navigation. At the same time, there are innovative applications and systems such as search recommendation, user touch, and understanding of live broadcast content in end-to-end cloud collaboration with huge application scale.

• Job Wanted:

  • Algorithms: CV/CG/recommendation/search/machine learning/model compression
  • Engineering: iOS/Android/Java server/C++ / on-end high performance computing

• Resume: [email protected]