How to make every lovely engineer work less overtime? Zhang Guannan, alibaba’s technical expert, has rich experience and experience in quality assurance system construction, continuous integration, agile practice and R&D effectiveness. Today, Guan Nan will use the actual case of Ali R&D team to vividly illustrate how to use data to drive the improvement of R&D efficiency.

This paper is the result of my practice in a business division with dozens of team members by using the cloud effect public cloud measurement function and the method guidance of the agile part, hoping to give you some reference. I’ll cover a variety of key characterization data, but detailed data, including specific r&d team data, will also need to visit the Cloud Effect public cloud Metrics page.

Note: The picture data in this paper comes from the cloud efficiency measurement data function.Interested children can click on the bottom of the article “read the original” to learn more.

The data show

Just to give you the numbers, I joined the team in April. Focus on the team’s march numbers:

Problem analysis

The obvious characteristics of this team are:

  • The number of completed requirements increased significantly in March, and the team was heavily loaded.

  • Low quality – number of defects, reopen rate and online release success rate.

  • The average requirement completion time is extremely long.

  • Surge failure.

Therefore, we took these problems exposed by the data and communicated with the front-line R&D staff, PD and TL to analyze the meaning behind the numbers. It was quickly agreed that the team’s main problems were:

  • In the traditional waterfall model, it takes one and a half to two months to complete a particularly large demand, but in the end, it deviates greatly from the expectation of users. In terms of data representation, the number of demands was small before, but a lot of demands were suddenly completed in March with a long time.

  • Everyone works overtime, the load is heavy, and more defects are introduced. PD and users are not satisfied with the modification, which will increase the workload, and this is a vicious cycle.

  • Defects are not highly valued, the management is not standardized, the priority division is not clear, and even the residual important defects are left in the bug list unsolved and flow online to cause faults.

The above three points form a vicious circle, resulting in more and more, more and more mistakes, more and more mistakes, more and more changes.

Solution landing and data operation

Once problems are found, it is relatively easy to solve them in a targeted way. Our solutions to the team are as follows:

  • Demand refinement: divide it into the minimum deliverable output, and try to avoid a demand that takes more than a month to be accepted by PD and users.

  • Embrace the user at all times: iteratively produce, deliver as acceptance, minimize inaccuracies, and correct errors when they are minimal.

  • Focus on quality management and operations: transparent data, encouraging the team to fix bugs as soon as possible, and strict pre-launch bug resolution standards.

  • Do your best to ensure successful online publishing.

At the same time, we assisted the team in making decisions. We conducted regular data operation, and made statistics and analysis of data every week, including those related to quality and efficiency, to ensure that we could find problems and correct deviations in the first time. So for more than three months, I focused on the following data. As for the interpretation and analysis of these data, the content is quite in-depth, so I only make a brief introduction here:

  • Throughput of requirements: The number of requirements completed by the team in a given period of time, which gives a rough indication of the team’s output trends.

  • Average requirement completion time: the average time from creation to final state of the requirement. The more time, the smaller the granularity of the requirement delivery, the higher the efficiency.

  • The number of newly added defects: the number of newly assigned defects in the team is counted, and the quality of team products and the efficiency of defect solving are reflected in combination with the existing defects and the average defect solving time.

  • Average defect resolution time: the average defect resolution time from creation to resolution, which represents the efficiency of defect resolution.

  • Success rate of online release: the ratio between the number of successful online release and the total number of online release, the higher the product quality is.

  • Reopen rate of defects: the ratio of the number of defects being reopen to the number of defects, the higher the value, the worse the quality of repaired defects; reopen rate is an important indicator of product quality.

Results analysis and summary

Let’s go back to the 6 graphs above and the defect resolution time chart below. We entered at the end of March, focusing on the data from April:

  • The team’s load was kept under control and the requirement completions dropped, remaining relatively flat for the next 3 months.

  • With the breakdown of requirements, delivery times dropped and the team delivered requirements to users at a faster rate.

  • The number of defects declines, the reopen rate declines, the success rate of online release increases, and the quality is improving.

  • The average defect resolution time increased significantly, the team delivered faster, reported problems faster, and resolved problems faster.

In general, it means faster delivery of requirements, faster feedback, lower cost of error/defect correction, slower convergence of defects, improved quality, and faster defect repair, which is a virtuous cycle. In summary, efficiency is improved, and quality is guaranteed. Team members work harder too!

How to further improve?

According to the demand quantity and average completion, according to data from the long team still have upside, for delivery on the size and rate of demand, or slightly fluctuation, if you want to know if what we do is faster, to meet the needs of users will be rapid, iterative delivery requirements, in order to avoid users want a car, we gave him four wheels.

Therefore, whether the delivery of the team’s requirements and the deviation of the user’s expectations can be completely solved, or we need to go one step further and refine the requirements to improve the delivery rate. See agile’s recommendations for fast iteration, fast delivery, fast user feedback, just to be faster and more accurate.

conclusion

Data is fascinating, as is R&D data, and we use it for two purposes: quality assurance; The second is to ensure the rate of delivery. During the walking process, the new function of cloud efficiency measurement is deeply used, which is combined with some ideas in Agile and traditional testing methods to help the r&d team.

May some people will question, need to use such cold numbers to measure our lovely programmer brother? My answer is: it’s not a measure. Data is just a tool, an effective tool to help us diagnose teams. Learn to use it and harness it. So we just need:

  • Pay attention to the data, read the data.

  • To focus on one or a few problems at a time.

  • I believe in the self-driven ability of the team, and combine TL management and motivation to develop a good team building ability.

Welcome to exchange and discuss

R&d team to deal every day is, release and application requirements, defects, code, test, and so on, these developers is closely linked with us data, cloud effect to develop the market, the team now space, staff performance, quality, distribution and other dimensions on the data platform for data integration, further more will be in the form of customized to meet the needs of research and development team for research and development data. Making good use of this tool can help us clearly understand the current situation of the team, expose problems, find improvement measures, improve team efficiency and product quality.

I am an Agile enthusiast, and I have learned some agile ideas when I went into the R&D team to do testing and quality management. My feeling is that taking the most practical tools such as siteboards, kanban, and fast iterative delivery requirements, coupled with data assistance, can help teams deliver high quality products faster and more accurately.

Finally, I will post some data of a RESEARCH and development team that I cut from the measurement. This team is a team that we have contacted recently. Through the data, we can infer that the team needs to improve its quality and strengthen its defect management. First of all, the number of team defects is increasing month by month, which is already reflected in the trend of poor quality.

In addition, the time to solve defects is not accelerated, which will lead to more and more defects flowing onto the line. It can be seen that the team has no faults except in January, and has faults in the following months. Moreover, the team’s online release success rate continues to be low, and the developer has low control over the code that goes live. So finding out what’s behind these data representations, and fixing them, is the team’s most pressing task in the near future.

Forming good r&d habits and maintaining efficient teamwork should be the persistent pursuit of every R&D student.

What have you learned about efficient development? Welcome to discuss and exchange in the comments section.

You might like it

Click on the image below to read it

How far away is unmanned operation and maintenance?

Ali Group big data in-depth interpretation

Computer vision top ICCV paper interpretation

Focus on “Ali Technology”

Grasp the pulse of cutting-edge technology