By Lo Ho
Founded in January 2015, Mikey is a mobile Internet technology company committed to achieving high-quality self-growth by taking digital leadership as its advantage. Always adhere to the user value as the center, data-driven, for users to develop rich tool applications, casual games, puzzle, sports and other mobile applications. More than 400 types of products have been developed, with more than 700 million users downloaded and installed. And over the next three years, Michael Jr. to become the world’s leading developers to increase service platform for the vision and mission, given to hope that through standardized products and services, provide developers with full link solution, with technology + services omni-directional escort, combustion products continues to grow, help tools and casual game developers to improve success rate of the product.
In The company, the management style is flat, and each business team can independently choose to adopt more suitable technology stack and infrastructure. Therefore, three different computing platforms, ECS, K8s and SAE (Serverless Application Engine), coexist in the company, and all of them are running micro-service architecture. Each computing platform has its own unique advantages and values, but it also has its own challenges, and SAE is currently being used by game teams.
Why SAE
For most casual games, the first thing is that the game itself has a life cycle, and within that life cycle, the game itself has very large peaks and troughs. For example, the flow during the day is much larger than that at night, and the flow during the day is concentrated in several time points, while 8 o ‘clock at night is the peak of the business, and there is almost no flow from 2 o ‘clock to 6 o ‘clock in the morning, but the service cannot be stopped. In addition, when the game was just launched, each operation activity would attract a large number of new customers, which required the background service to quickly respond to the change of traffic, so the business side expected to have an automatic elastic and scalable computing platform. Secondly, most casual games are stateless, and can be divided into different service modules to improve service performance and quality, such as chat, red packets, backpacks, upgrades, user data acquisition, video processing, advertising, etc., so they can be deployed using microservice architecture. Finally, during the launch of the game, many new functional modules will be added iteratively, requiring frequent release upgrades. Therefore, when selecting the model, the business side will consider comprehensively:
- System stability and disaster recovery capability
- Automatic elastic expansion of the platform
- Support for microservices architecture
- Easy release rollback capabilities, and even constant updates
In fact, these capabilities can be achieved through ECS or K8s self-built, but will bring a lot of operation and maintenance costs to the business team, and it is difficult to balance the cost input. Especially in the aspect of elasticity, self-built elastic efficiency is difficult to meet the rapid change of flow, and a large number of redundant resources are often needed. SAE can meet these four requirements very well. In fact, Mikey’s game team has been interested in testing SAE since the open beta period. So far, more than 50 services and applications have been deployed on SAE, involving more than a dozen games, such as love to guess Idioms, The strongest Answer to Idioms, I Find Fault fast, Daduo, Happy Find Fault, Duoduo short video, etc. If you are interested, you can download the APP and try it.
SAE Landing Practice
Serverless application Engine SAE is positioned as a one-stop application hosting platform based on containers. Its core value is to provide users with a K8s operating environment with full application lifecycle management, micro-service governance and flexible o&M. In essence, the user’s code still ends up running in the container, but the container doesn’t have to be managed. As a result, existing game services can be migrated directly to SAE without modification. In addition, SAE also provides a JAR package deployment mode for JAVA applications, which eliminates the step of Micky mirroring. It is very similar to the original MODE of using ECS, but the experience will be simpler. The general comparison is as follows:
The core capability of SAE is high availability and automatic flexibility. For Mikey’s game team, multiple availability zones can be checked when deploying JAR packages to achieve cross-availability disaster recovery. The SAE infrastructure actually provides multiple K8s clusters distributed in different availability zones, and the container instances hosting the business can be automatically scheduled in multiple availability zones. The elastic configuration is also very simple, which can be set based on CPU, memory, QPS, RT and other indicators. For Mikey’s online games, expansion and contraction can be triggered mainly by CPU and memory usage, and the maximum and minimum instances can also be specified, which is very convenient. At present, timing elasticity and monitoring indicator elasticity can be used together. In case of operational activities, the two elasticity modes can be used together to ensure resource elasticity. Note that the threshold of the monitoring indicator needs to be set based on the actual service situation. You are advised to determine the threshold by performing a pressure test before going online.
In addition, through application monitoring, it is very convenient to check the invocation of the service interface. These capabilities have been integrated into the SAE platform by default, which is very helpful for business troubleshooting.
Finally, the game team of Mikey mainly adopts Spring Cloud and Dubbo technology stack, so it is very necessary to support microservice governance ability. At present, SAE console can directly configure microservice health check, elegant offline script, configuration management, microservice gray publishing, one-click rollback, etc. However, in the process of actual use, I have stepped on some pits. For example, in the process of service release, the health check sometimes times out and the instance restarts constantly, because the service sometimes loads a large amount of data and class libraries, which takes time to start. Increasing the health check timeout reduces the probability of occurrence, but the release time will be longer. In addition, the initial response was slow when the service was just started. In fact, the service was not fully ready, so SAE was more dependent on its ability to provide the elegant launching of micro-services to ensure the normal launching of services. In addition, for the fractional wholesale part, in order to avoid the sudden transfer of load traffic to new instances, it is recommended to use the grayscale capability of micro service traffic percentage. After a period of practice, the final business architecture is roughly as follows:
The game team of Xiao Mai basically only focuses on business logic, and the resource level is hosted by SAE platform, which greatly simplifies operation and maintenance. In addition, in order to cope with the rapid business iteration, Jenkins also encapsulated SAE API interface to realize CI/CD capability, which greatly accelerated the service on-line speed. Compared with the original elastic efficiency and deployment efficiency, the overall r&d efficiency has been greatly improved. The elastic speed has been shortened from minute to wonderful level, and the online speed of new projects has been shortened from day to minute level.
Summary and Outlook
1. SAE provides a Serverless operating platform in the field of microservices, providing users with a new choice to reduce costs and increase efficiency. In addition, the hosted K8s cluster at the bottom of SAE also provides the easiest way for users to do container transformation.
2. SAE’s addition of application management and microservice governance makes SAE a one-stop application PAAS platform that is different from container services, allowing users to focus on business iterations.
3. For application management, SAE also provides the environment “one-click start and stop” function. For example, for the development and test environment, it can set the timing to close and open, optimize the resource occupancy of the off-line environment, which can help optimize the cost of a small step forward.
4. For JAVA applications, SAE provides a version of the DragonWell JDK that speeds up JAVA application startup and thread resource consumption by approximately 40%.
5. SAE will continue to improve flexibility and enhance functionality iteration at the application management level in the future, with the expectation of adding value to the user experience. For example, THE new PHP-enabled ZIP package deployment capability will simplify cloud complexity in WEB applications.
For those of you who are interested in Serverless, click here for more cases and articles.