preface
As an excellent one-stop mobility and living platform, Didi provides taxi, express, private car, luxury car, bus, designated driving, enterprise, shared bike, shared e-bike, car service, food delivery, payment and other diversified services to more than 550 million users in Asia, Latin America and Australia.
Tens of millions of car owners and drivers have access to flexible work and income opportunities on didi’s platform, carrying more than 10 billion passengers a year.
In its three years of open source, Didi has adhered to the concept of “embracing openness, win-win cooperation and creating value” under the initiative of Dr. Wensong Zhang, senior vice president of Didi and chairman of the Open Source Committee.
Didi actively participates in open source projects in the industry and constantly exports its excellent internal projects. Since the release of Didi’s first open source project VirtualAPK on June 30, 2017, Didi has released more than 40 projects, which earned 6.3 million stars and received a good response from the community. Here’s a quick look at Didi’s 40+ open source project. (In alphabetical order)
Artificial intelligence
AoE
AoE, named after AI on Edge, is a terminal side AI integrated runtime environment (IRE).
AoE, based on the design principles of “stability, ease of use, and security”, helps developers easily deploy deep learning algorithms of different frameworks to terminals for efficient execution.
Project Address:github.com/didi/aoe
Athena
Athena is an open source implementation of the end-to-end Automatic Speech Recognition (ASR) engine.
Currently, the project supports connector time classification (CTC) based models, transformer-based codec models and hybrid CTC/attention-based models as well as mpC-based unsupervised pre-conversion training and decoding.
Our vision is to enhance end-to-end models of speech recognition for industrial applications and academic research. To make ASR available to all, we have also released sample implementations based on open source datasets,
For example HKSUT, Librispeech all of our models are implemented in Tensorflow> = 2.0.0.
Project Address:github.com/didi/athena
Chinese NLP
Chinese Natural Language Processing (NLP) related links: Datasets and recent results.
Project Address:Github.com/didi/Chines…
Delta
DELTA is a deep learn-based speech and natural language understanding modeling platform designed to create an industry-wide speech and natural language understanding modeling framework that is easy to use, simple to launch, and quick to develop.
Project Address:github.com/didi/delta
DLFlow
DLFlow is a deep learning pipeline that combines Spark’s large-scale feature processing capabilities with Tensorflow model building capabilities.
DLFlow can quickly process original features, train models and make large-scale distributed prediction, which is very suitable for off-line production tasks.
Project Address:github.com/didi/dlflow
HetSANN
Source code for HetSANN in AAAI’20 paper: An Attention-based Graph Neural Network for Heterogeneous Structure Learning.
Project Address:Github.com/didi/hetsan…
maskdetection
To further assist in combating Novel Coronavirus, Didi Chuxing has decided to make its mask detection technology available to the public for free.
Mask detection technology developed by The DiDi AI team is based on DFS face detection algorithm, and DiDi has adopted facial attribute recognition algorithm on its platform.
The model overcomes some difficulties, such as complex daytime lighting changes, facial posture changes, facial proportions, etc. It uses weighted loss functions and data enhancement methods to deal with different mask types and uneven mask data during the day and night.
The system can use uploaded images to identify unmasked drivers with an accuracy of 99.5 percent, and can achieve 98 percent accuracy during on-site inspections of DiDi’s on-board cameras.
The model was trained on a dataset of 200,000 faces to ensure its robustness.
This quick detection system can be used in a wide range of travel scenarios, including mobile phone photos, surveillance images, and can work 24/7.
Project Address:Github.com/didi/maskde…
SQLFlow
Didi DS statistical science team based on the needs of common internal business scenarios, and ant Financial SQLFlow initial research and development team from the model, community, and culture of all-round cooperation. The two sides complement each other in capabilities and have strong synergy in resources, aiming to promote the use of AI by anyone who knows business logic and make AI accessible to those who know the most about business.
Project Address:Github.com/sql-machine…
Intelligent Transportation
ALITA/ALITA-UI
ALITA (A Layered Instrument To Analyze) is A Layered data analysis tool produced by the Data Department of Xiaojuche.
Through a large number of business requirements, we realize precipitation and abstraction of the universal “point”, “line”, “surface” three data models, and the map rich point, line, surface design elements perfect integration, design and packaging of the front and back end universal modular components, to achieve rapid map application.
Project Address:github.com/didi/ALITA
mtmc-vt
This code is for AI City Challenge 2019 Track1, MTMC Vehicle Tracking.
Project Address:Github.com/didi/mtmc-v…
TrafficIndex
TTI is a widely used evaluation index of urban congestion degree in the industry. It reflects the ratio relationship between the travel time actually spent and the travel time spent by free flow. The larger the value is, the worse the traffic operation state is.
Generally, it is positively correlated with congestion degree. Other factors, such as abnormal weather (such as rain, snow, fog, etc.) or abnormal road conditions, may also affect TTI value.
Project Address:Github.com/didi/Traffi…
3. Middleware and architecture
Booster
Booster is an easy-to-use, lightweight, and extensible quality optimization framework designed specifically for mobile applications, providing extensible capabilities through dynamic discovery and loading mechanisms.
It is not just a framework, but a rich set of quality optimization tools built in.
Project Address:Github.com/didi/booste…
DDMQ
DDMQ is a message queue product built by Didi Chuxing Architecture Department based on Apache RocketMQ.
As a distributed message middleware, DDMQ provides message services with low latency, high concurrency, high availability and high reliability for each business line of Didi Chuxing.
DDMQ provides a variety of message types including real-time messages, delayed messages, and transactional messages to meet different business requirements.
Users can easily access DDMQ to produce and consume messages and experience a rich and stable messaging service through a unified Web console and a foolproof SDK.
Project Address:github.com/didi/DDMQ
Go-Spring
Go-spring’s vision is to make a programming framework as powerful as Java Spring available to GoLang programmers.
Features: provides a complete IoC container, support dependency injection, attribute binding; Provide a powerful starter framework, support automatic assembly, out of the box;
Provides an abstraction layer for common components, enabling flexible replacement of underlying implementations; Go-spring is currently developed using Go1.12 and dependency management using Go Modules.
Project Address:Github.com/didi/go-spr…
Big data
ES-Fastloader
Es-fastloader is a solution that can quickly generate indexes for massive offline data. It adopts a Build & Push mechanism.
With the powerful parallel computing capability of Hadoop, dozens of TERabytes of data can be constructed very quickly within 1-2 hours.
Solve the problem of low timeliness of index file construction when constructing ES with massive data.
Project Address:Github.com/didi/ES-Fas…
Levin
Levin is a fast loading solution for low-frequency updates, static use, and large-scale data.
Levin implemented a set of STL-like containers used on SHM shared memory fragments to efficiently host large static data and speed up cold startup and hot loading of large memory services.
It has the advantages of easy to use, high efficiency, good performance and memory saving.
Project Address:github.com/didi/levin
5. Operation and maintenance monitoring
Elastic-trib
Elasticsearch is a command line tool for Elasticsearch cluster management. It is used to manage more than 30 Elasticsearch clusters in an enterprise.
It can be integrated into shell scripts and used for cluster management and control through command line terminals.
Project Address:github.com/ didi/elastic-trib
falcon-log-agent
Falcon-log-agent falcon-log-Agent is an open source log collection tool designed to capture and collect feature information from streaming logs.
The characteristic information obtained was connected with the Open-Falcon monitoring system.
It can be used to measure business indicators and build stability.
Project Address:Github.com/didi/falcon…
Kafka-Manager
One-stop Apache Kafka cluster indicator monitoring and operation and maintenance management and control platform.
Project Address:Github.com/didi/kafka-…
Nightingale
Nightingale is an enterprise-level monitoring solution developed by Didi Basic platform and Didi Cloud and open source. It aims to meet the needs of enterprise monitoring in the cloud native era.
Nightingale meets enterprise-level requirements in terms of product completion, system availability, and user experience.
It can meet the scenarios of users of different sizes, from a few machines to hundreds of thousands of perfect support.
Both cloud native and bare metal, support application monitoring and system monitoring, flexible plug-in mechanism, plug-ins rich and perfect, with a high degree of flexibility and scalability.
Project Address:Github.com/didi/nighti…
sqt
This process is used in Didi Cloud to manage other agents on the machine, such as monitoring agents and security agents.
The management is mainly: install, upgrade, uninstall, check the startup status, do not do other things. Customers need to install other agents manually.
Project Address:github.com/didi/sgt
6. Small programs
Chameleon
CML, as a framework for truly allowing a set of code to run multi-terminal, provides a standard MVVM mode for unified development of various terminals.
At the same time, it has independent runtime framework (Runtime), data management (Store), component library (UI), interface (API).
In addition, CML has done a lot of work in strengthening cross-end capabilities, unifying capabilities, and performing consistently.
Project Address:Github.com/didi/chamel…
Mpx
Mpx is an enhanced applets framework dedicated to improving the development experience and efficiency of small programs. Through Mpx, we can efficiently and elegantly develop high-quality applets with extreme performance, and export them to various program platforms and web platforms.
Project Address:github.com/didi/mpx
Mobile development
Echo
Echo is a debugging tool that is easy to use, easy to expand, and easy to display and operate on a large screen. It aims to improve the development and debugging efficiency of mobile terminals.
Based on some existing problems and team needs, we developed a desktop debugging tool Echo, which can help us view all kinds of App data (network requests, logs, buried points, etc.) in real time, and can also quickly modify the UI effect of debugging App without changing the code, thus improving our development and debugging efficiency.
Project Address:github.com/didi/echo
DoraemonKit
DoKit, the Chinese name for Doraemon, means being able to provide A variety of tools to his owner like Doraemon.
Is a fully functional client (iOS, Android) R & D assistant. Can let each App quickly access some commonly used or you do not implement some auxiliary development tools, test efficiency tools, visual AIDS.
And the perfect way to plug into the Doraemon panel some of the non-generic assistions that you’ve already implemented that are tightly coupled with your business, and with our DoKit platform, make the functionality extended, easy to access, easy to expand.
Project Address:Github.com/didi/Doraem…
DroidAssist
DroidAssist is a lightweight Android bytecode editing plug-in that enables dynamic modification of class files through simple configuration in XML.
Project Address:Github.com/didichuxing…
VirtualAPK
VirtualAPK is an excellent plug-in framework developed by Didi Chuxing. By plug-in business modules, the plug-in can be updated at any time to release new functions, and the version can be released at any time.
This plug-in framework can help enterprises to release new functions by updating plug-ins at any time, including repairing serious Crash or carrying out business “trial and error”. It has four core advantages: complete functions, basically no intrusion, easy access to host code and resources by plug-ins, and high compatibility.
Project Address:Github.com/didi/Virtua…
System tools
Collection
The Collection package is intended to replace Golang’s native Slice in a large number of scenarios that do not pursue extreme performance and pursue business development efficiency.
The Collection package currently supports the following elements: int, INT64, FLOAT32, float64, String, struct.
Project Address:Github.com/didi/collec…
GateKeeper
GateKeeper is a distributed database-independent API gateway written in Go (Golang) that enables efficient service proxy and online service configuration without restarting the server.
Project Address:Github.com/didi/gateke…
Gendry
Gendry is a Go package for assisting with database operations. Based on go-SQL-driver /mysql, it provides a series of methods to prepare parameters for calling methods in the standard library Database/SQL.
Project Address:github.com/didi/gendry
JuShaTa
JuShaTa is a Java container that provides module isolation and module hot loading capabilities. We provide a Java container JuShaTa similar to Tomcat, in which each SpringBoot service is an independent module.
By customizing classLoaders, different modules use different classLoaders to load and resolve JAR package conflicts. Spring Context is used for Context isolation, one Context per module, to resolve bean conflicts.
Project Address:Github.com/didi/JuShaT…
Kemon
A framework for macOS kernel monitoring based on open source back-and-forth callbacks. Kemon is a MacOS-based kernel monitoring framework that has a good foundation for kernel monitoring on Windows.
However, there is no mature hook framework on the Mac to track the generation of more process kernel events. Kemon is further improved to make due contributions to the improvement of macOS kernel security.
Project Address:github.com/didi/kemon
SDS
SDS (Service Downgrade System) is a lightweight, simple and easy-to-use streaming, fusing, and Downgrade System. It can automatically Downgrade, fuse, and quickly recover Java applications and improve the overall “elasticity” of applications.
At present, the server can cope with the complex and heavy traffic scenarios by adopting the popular micro-service architecture, and can still achieve strong rapid iteration ability and scalability when the business is developing at a high speed.
Micro service architecture is not the whole system will be more simple, on the contrary, micro service architecture management difficulty is higher than the ordinary centralized architecture, so, how to ensure that the system of each node in a complex environment can provide the service stability, need to use tools to make service node can resist the traffic impact, fusing dependence bad points.
Project Address:github.com/didi/sds
Tinyid
Tinyid is a distributed ID generation system developed in Java. It is based on the database number segment algorithm. For this algorithm, please refer to The principle of Meituan Leaf or Tinyid.
Tinyid extends leaf-segment algorithm, supports multiple dB (master), and provides Java-client (SDK) to localize ID generation, achieving better performance and availability.
Tinyid is used in Didi’s customer service department and is accessed through TinyID-client, generating ids of 100 million levels every day.
Project Address:github.com/didi/tinyid
The front end
cube-ui
Application OF Vue to build an excellent mobile UI library tools. The technology has four characteristics of reliable quality, extreme experience, standard specification and strong scalability, and has a unique post-compilation technology solution to help significantly optimize performance.
The goal of Cube-UI is to make mobile development easier, make developers more focused on business logic development, and improve r&d efficiency.
Project Address:Github.com/didi/cube-u…
di18n
Di18n is an automatic transformation, configuration – based front-end internationalization solution. It can automatically scan the Chinese text in the code and replace it with internationalization markup; At the same time, the language is extracted into the configuration, which can be saved and updated on the server.
Project Address:github.com/didi/di18n
Epage
Epage is a schema-based visual page configuration tool. Abstract from work order system flow form scene, upgraded to support cross-frame, component library rendering, customizable universal page configuration tool. It is often used for process form, middle and background page configuration.
Project Address:github.com/didi/epage
Mand Mobile
Mand Mobile provides 30+ utility components that meet most of the requirements for Mobile page development. The business component is also targeted at the financial field, including the extraction of charts, numeric keypads, so as to better meet the development needs of related products.
Project Address:Github.com/didi/mand-m…
Mand Mobile-RN
Mand Mobile RN is short for Mand Mobile for React Native. Mand Mobile’s React Native component library, while focusing on financial scenarios, also provides components and solutions unique to RN projects that improve cross-end development efficiency and UI experience.
Project Address:Github.com/didi/mand-m…
Pile.js
React builds a lightweight mobile component library. Efficient coordination between design and front end, fast building of mobile component library.
Project Address:Github.com/didi/pile.j…
Tips
Tips is a static copywriting management platform. Static copy for modifying Web pages, supporting internationalization of copy, and providing UI presentation of prompt information.
Its purpose is to solve the problem of frequent static copy modification by front-end developers, and avoid the complicated online process initiated by simple copy modification.
Project Address:github.com/didi/Tips
Research and development testing
benchmark-thrift
Benchmark-thrift is an efficient and simple tool for testing thrift applications out of the box.
Project Address:Github.com/didi/benchm…
Rdebug
Rdebug is a practical tool for RD research and development, self-testing and debugging developed by Didi. It can be used to improve RD development efficiency, ensure code quality and reduce online accidents.
Project Address:github.com/didi/rdebug
Sharingan
Sharingan is a golang-based traffic recording and playback tool, which is suitable for project reconstruction and regression testing.
With the rise of microservice architecture, the dependency relationship between services becomes more and more complex. Frequent system upgrades lead to high cost of maintaining test cases, and it is difficult to provide a stable test environment with numerous downstream dependencies. Therefore, we develop this set of tools to alleviate the above problems.
Project Address:Github.com/didi/sharin…
thrift-mock
Thrift – Mock is a lightweight Java test tool that simulates the Thrift service. Using thrift, you can easily mock the thrift service interface to get the specified interface return, which greatly improves the development efficiency in the joint debugging and testing stages.
Project Address:Github.com/didi/thrift…
Source: Didi Technology
Give a [look], is the biggest support for IT elder brotherCopy the code