Important note, this article will only provide key terms, not too much technical details, technical details to dig yourself.

Faster file transfer and file copy in Python – Giampaolo Rodola

The first speaker on stage was Giampaolo Rodola, a Core Python developer from Italy, who wanted to be awkward, but stopped short because of his poor English. He didn’t know every word at the conference, but he did know what he was sharing.

His sharing comes in two chunks.

Part 1 shows that the underlying copy in Python3.8 uses the os.sendfile() or socket.sendfile() method to copy files, which is faster than the old copy method.

The operating system is divided into user mode and kernel mode. The old method of copying files switches between user mode and kernel mode several times, while os.sendFile () does all operations in kernel mode, so it is faster.

The second part mainly introduces pSUTIL, through pSUTIL can realize most of the computer state monitoring, such as CPU, memory, disk, process, network, etc., using PSUTIL can actually build a computer state monitor.

Google SRE System core basic interpretation — Liu Zheng

The second person on stage was Liu Zheng (Elastic AD), who was talking about Google’s SRE(Website Reliability Engineering). SRE has three things: SLA, SLO, AND SLI, and when I was listening to the meeting, I thought it was more like one way of working. If you are interested, you can search for it

Make money from Python — See Zehui

KuangZeHui teacher major share his personal hobby projects, how to use the Python copy virtual money to make money, for I made for a period of time quantitative, in this share is not particularly strong logic support, mainly using the grid strategy, this strategy theory in the large fluctuations in the market will have a good effect, and the virtual currency is a big fluctuations in the market, The core logic of grid strategy is to make use of fluctuations to buy and sell, buy when falling, sell when rising, so that the average price of buying when falling multiple times is less than the average price of selling when rising multiple times minus the average price of transaction fees.

The project shared by the teacher was somewhat arbitrary, and some points seemed illogical and inconsistent. But the key to the sharing was not the project itself, but the idea of the unity of knowledge and action. The unity of knowledge and action came from The Yangming mind theory of Wang Yangming, who realized enlightenment in Longchang after being frustrated and proposed the unity of knowledge and action. (Wang Yangming’s book is worth a look, I was the technical manager of the board company before the pit finished reading).

A line of code speeds up scientific computations — solutions

Modin library is a library that can be accelerated to pandas by one line of code.

Pandas as pd is changed to import modin.pandas as pdCopy the code

Modin implements 70% of the Pandas API and is used in exactly the same way as Pandas.

Pandas is slow because it can only use a single CPU core. Yes, but they didn’t implement the logic to support multi-core usage. The main improvement of Modin is that it can use all the CPU resources in the device to achieve a huge speed increase.

But does Modin have a good community? It’s very, very painful to have an imperfect community and a pit.

Modin Project warehouse address:

Github.com/modin-proje…

Digital Currency Transaction System Architecture and Python Implementation — Huang Yi

After the meeting, I had an awkward chat with Teacher Huang Yi, who has profound skills. I didn’t have an in-depth study of Redis, so many things were beyond the scope of knowledge.

Teacher Huang Yi shared the architecture of the digital currency trading system he constructed. Different from the traditional Web system architecture, the matching function of the trading system must be executed in the global order. The so-called matching is to find the lowest price of the seller and the highest price of the buyer, so that the two parties can trade.

You need to find the buyer in the whole system, and then find the seller in the whole system, and then do the transactions in order. This part of the transaction is not concurrent, that is, you can’t do multiple transactions at the same time, because each transaction affects the next transaction.

This puts a theoretical ceiling on the capacity of the system and makes it impossible to scale the system by adding machines to build clusters. (I am very curious about how the A-share trading system and the US stock trading system solve this problem.)

Huang Yi’s solution is to use the Redis Module to achieve a new Redis data type to meet the business logic and make the business logic complete in memory. Of course, the data is persisted in a relational database (Ticks: SQL stitched together into batch operations to increase the speed of inserting data).

The trading system relies on Redis Module and Redis Stream(Redis 5.0 +, similar to Kafka) and can achieve 100,000 transactions per second on a single core (shout 666).

The big guys share their open source streaming computing work: github.com/cryptorelay…

Their company Crypto also has recruitment, the salary is not generally high, Python development: 50K-100K (experience: 5-10 years).

Application of Python AI development in Microsoft Cloud — Jianhui Lu

Professor Lu Jianhui – Microsoft’s most valuable expert

Maybe it was because of the teacher’s deep knowledge that I did not Get the charm of the speech. Although THE topic of the speech was artificial intelligence, most of it was about Azure, namely advertising.

Because I have done NLP for a period of time, I generally know that deep learning is not friendly to individual players at present. Currently, the training of well-known models requires a huge amount of computing power, which is equal to money and requires clustering. This kind of free training on the cloud can only make some simple models.

Of course we can use transfer learning to use models that others have already trained, but that has nothing to do with sharing.

Also interesting is that VS Code supports The Jupyter plugin, which makes it easier to use Jupyter directly in VS Code.

VS Code is developed by Electron, which is open source itself, and its Code is worth learning. I recommend a blog from a leading man of Taobao, who has been working on the transformation of VS Code to form his own development tool, called Editor, for Taobao. His blog is as follows:

www.barretlee.com/blog/2019/0…

FPGA aided Python acceleration calculation — Chen Zhiyong

Teacher Chen Zhiyong mainly shared the FPGA which can be semi-customized circuit, using FPGA+ programming can achieve some interesting effects.

At the beginning, it mainly introduces the knowledge of FPGA hardware. The parallelism of hardware is realized by using multiple electronic components, while a single electronic component can only realize the effect of concurrency.

In addition, functional programming language was also mentioned. Teacher Chen Zhiyong said that functional programming thinking must be used when writing programs on hardware, because I only used Erlang, a functional programming language for game development, so I didn’t quite understand this sentence.

The only Python-related part of this sharing is the PYNQ library, which uses the PYNQ API to write programs that run on the company’s hardware and are fast because PYNQ maps the corresponding Python language to RTL code on the hardware for extremely fast computing speed.

In addition to using Python, you can also use C to write, using Vivado HLS this tool, you can convert THE C language into RTL code, the conversion process should be the use of the compilation principle related knowledge, but the conversion effect is not the use of PYNQ this mapping into RTL way good, The reason is that the WRITTEN C language does not use the idea of hardware development to write the code, and the RTL code converted at this time is not well written, resulting in low efficiency. However, PYNQ loses flexibility to achieve the normalization of mapped RTL code.

In fact, many tools will have a variety of problems in the way of code conversion, such as Unity development of the game to H5, H5 can indeed run, but the card is not good, mobile phone completely can not play, in addition to Debug and other problems are pit.

Pipenv and Python package Management — Minch

Minch shared some details about the Python virtual environment and package management, starting with the correct way to install packages and various pits.

Pipenv was then introduced, briefly describing dependency resolution problems and corresponding solutions.

Finally, we’ll talk about a possible future for Python package management, including PEP517 and PEP518, which is essentially node.js using NPM packages. Like NPM, you can choose to install dependencies globally or in the current project directory, in the current project directory. It will only be used by the current project, so you don’t have to worry about the virtual environment.

Then introduce PyFlow this tool, can achieve PEP517, PEP518 draft effect, address is:

Github.com/David-OConn…

However, NPM has its own problems, and in 2019 JSconfEU presented Tink’s next generation package manager technology, intended to replace NPM.

The lightning speech

Each talk lasts about 10 minutes

Python C extensions package and release on various platforms — Feng Zhao

In Continuous Integration (CI), you can automatically package Python C extension libraries.

Unlike building pure Python packages, building Python C extensions involves a compilation process. In Linux, the C extension library can not be uploaded to the official. Here Python official gives a Centos6.1 environment Docker, must be in this Docker Linux package to upload.

Automatic Web UI generation for Python Function — Peng Weikang

Teacher Peng Weikang introduced his own developed tool touch-Callable, which is built on Flask. It can quickly build a Web interface by one method, which is easy for testers to use. The effects are as follows:

# examply.py
from enum import Enum


classswitch(Enum):Open ='开'Close the ='off'


defWater dispenserPassword: STR, heating: switch =None, Cooling: switch = switch('开')):
  """ This is 20618!" ""
  ifPassword! ='Drink more hot water':
    raise ValueError('Who are you? I don't know you.')

  # omit specific logic
Copy the code

Github:github.com/pengwk/touc…

Digital Currency Transaction System Python practice — Dai Shaofei

It introduces the problems encountered in developing the trading system and provides solutions, which are surprisingly simple and do not involve any high technology.

In addition, APScheduler is also introduced as a scheduled task library (because it is used in their system). This library has a lot of concepts, if you are interested, you can have a look:

Github.com/agronholm/a…

However, it is not recommended to use apscheduler for simple scheduled tasks because it will increase the complexity of the system. Crontab is simple and simple.

Django implements backend low code development platform — Jeff

The first stage is to write a small amount of code, like Django Admin, to implement web functions. The second stage is to build a Web system through configuration files. For example, to build a Web system from JSON files, the third stage is to build a Web system through interface configuration.

But I personally think the third stage is to build a Web system in a draggable way. Because the essence of interface configuration is still to generate a JSON file, if the logic needs to be changed, it still needs handwritten logic, which is not flexible enough. At present, the commercial low-code platform I know is to build a web platform without code by using a structure similar to logic diagram (many quantitative platforms actually have similar things, Such as bigquant).

Drag through the interface, will be more flexible, you can build a variety of logic, but the operation complexity is also larger, most of the time to build a Web, such as building background, through configuration to build better, because most of the time is to add, delete, change and check, there is no special logic.

The tail

This is actually the first time FOR me to parameter this kind of meeting, because BEFORE I thought that the meeting is not meaningful, what can I learn in a day? So I didn’t attend much, but this time I felt very good, met some new friends, broadened my horizon, that’s the meaning of the conference.

Finally, thank you for reading, if the content is helpful to you, please click “good-looking”, that is clickable, thank you.