At the end of 2014, I joined a startup company and was mainly responsible for the Web direction team.

To be clear, this is a super-fast, hard-hitting company. I don’t know, it means that you have an idea today and you have an idea tomorrow. Oh, no, not just an idea! Is the style that must be verified online tomorrow.

Under this super fast pace, the supporting front-end engineering configuration is very scarce, so we embarked on a section of engineering heart course.

Let’s take a look at the situation we were facing:

  1. The product is moving at a very fast pace. It’s going to launch tomorrow.

  2. The front end is only divided into TWO major projects, PC and Mobile. All the pages of these two platforms are here, and the coupling is serious

  3. Build and deploy test and online environments by hand.


Take a look at our development test launch process:

1. The front-end projects are isolated from the back-end through Ajax, so the development environment is completely localized. After the local joint adjustment, the test environment is deployed for testing.

2. Operation and Maintenance students are very generous to provide many sets of test environment (test1~ TEST100), for safety, there is a jumper between the test environment and the development machine (fortress) isolation.

3. Use Grunt test1 to TES100 to produce test packages (2 packages, 1 page loaf, 1 static resource pack), copy them to the Fortress, and then copy them to the corresponding directory of the target test environment (also 2 directories). In a formal environment, the grunt product outputs the formal package, then copies it to the jumper, and then copies it to the specified directory of the op specified machine. Then op(operation and maintenance students) operation online.

  

The above situation caused us to have a lot of trouble in schedule development, and a lot of time was wasted on deployment:

1. Serious code coupling, multi-person maintenance and unified project plus ultra-fast development and iteration rhythm, the final online thing can only be slash-and-burn, that is, the project can not be fully online, each online can only be manually picked out which files this time online

2. Manually copy the deployment data to the jumper and then to the target machine. Repetitive manual labor is heavy. With multiple test environments, deployment is even more painful.

3. Multi-person maintained projects often commit code that shouldn’t be committed or have trouble going live because local builds forget to update

4. Test environment Is often overwritten during the test, resulting in a waste of test time


To solve this problem, we made the following optimization steps that resulted in our deployment system, Ideploy.


Step 1: Replace manual SCP with deployment commands, reduce repetitive manual labor, and say no to slash-and-burn farming

As mentioned above, we did all of our initial build deployment in a slash-and-burn way, so let’s look at our deployment process:

First we need to build the code package to be deployed locally, then SCP to bastion, then log in to our jumper, then SCP to the target machine. It’s a good job once or twice, but it’s a nightmare every day.

Yes, a bit of an experienced person will wonder why we don’t use shell scripts to do this, which is exactly what we did immediately. We use the Expect script to automate the transfer of code from the native machine to the fortress machine to the target machine. We wrote a nodejs command-line program called wdfe to deploy the code to test1 by running wdfe deploy Test1 directly after the local build, similar to other environments. So we only need 2 lines of command per build deployment

1. npm run build test1

2. wdfe deploy test1

This command line program greatly reduces our manual labor.


Step 2: Build the platform first and solve the existing problems first.

With the command line program, our efficiency is greatly improved, but the command line program actually has some limitations, such as:

1. Maintenance issues of the command line program, every update (possibly the deployment target machine change, etc.) needs to ensure that everyone updates the version.

2. Only full updates are available.

3. Local builds introduce some out-of-sync code issues.

There are limitations that can be fatal in a team situation like ours. Due to the historical problems, for example, our project is a large project, many people maintain and update online several times a day that we can only go part of the file online this road (are not all engineering online), so wdfe deploy this whole project launch way in a short period of time we cannot be implemented (product rhythm, human are issues), This can only be used in test environments. But manually pick out the online file this way is indeed a common resentment, such as a more modified, someone modified dozens of files, then the person needs to manually pick out the dozens of files from the big project, the probability of error is very large. So when the line immediately after the things often happen. To solve this problem, we wrote a deployment system in NodeJS that lists which files have changed in the code repository since the last deployment. This reduces the amount of manual work by 90% by allowing the deployer to select the files to deploy. Here’s a selection of deployment files screen (we’ve kept this feature until now because it’s still possible to deploy a single file in a particular case) :

   

  

Step 3: Break up the project, improve the deployment platform, and make the deployment smooth.

Through the first two steps, we solved most of our team’s pain points on construction deployment through tools and platforms, and reduced a lot of repetitive physical labor. We moved on to the third step, which took the effort out of building the deployment.

First of all, let’s look at the engineering requirements of the team after the completion of the second part and our solutions

  1. Support full deployment, do not need to select deployment files, of course, also support partial file deployment (this is the engineering code organization problem, need to be solved by splitting the project, but splitting the project to fine granularity, more need deployment platform support, such as one click deployment of multiple projects and multiple machines)

    In view of this, we organized manpower to divide the project into multiple sub-projects, and each sub-project has no dependence on each other, so it can build and deploy by itself. And in the deployment to provide multi-machine, multi-project one-click deployment function.

2. After testing the test environment, there is a way to solve the problem of test environment coverage

First of all, close the deployment portal, all deployments must pass through the deployment platform, and then give the deployer an option after the deployment of the deployment platform, whether to lock the deployment from others, to solve the coverage problem:


3. Before going online, it is necessary to know that this online needs to be listed, and it is safe to go online

For multi-maintenance projects, we always want to confirm which files have been modified by those people during deployment to avoid omission. We submit logs and diff to make it very clear to the deployed students. In checkout step before deployment, we listed The Times of modification of each member, modification record and changes of each file between this launch and last launch, and we could detail each modification



4. Easily add projects and target machines.

Projects, adding and modifying deployment machines, are very simple, adding a project and a target machine only takes a few minutes. Breaking up projects, adding and modifying deployment target machines is completely effortless

5. Build and deploy independent, supporting the project to choose the technology stack freely.

The build is entirely up to the project, and the deployment platform is only responsible for invoking the build command and deploying the build results to the corresponding directory on the corresponding machine according to the rules

The deployment process of the deployment platform is as follows:



After this step, we basically formed our own deployment engineering architecture, and building deployments was no longer the burden of our team members. We’ve also added interesting modules like deployment statistics to give you a general idea of how our projects and people are deployed.


We actually made the platform open source and put it on Github under the name ideploy. , hope we can help to some of these practices like we have been or are experiencing the pain of company: https://github.com/wdfe/ideploy, you can enter the program by reading the original at the bottom of the page