What is artificial intelligence automatic programming? Is it just a gimmick?

There is no doubt that AI will change the way software is developed — we are already seeing the benefits of some attempts to apply AI to software development, such as “program code generation” : generating code from images, generating code from data models… What I’m interested in today is whether the average developer already has some useful tools that use AI techniques to increase his productivity.

I looked into the realm of common programming, searched for tools I call “intelligent programming,” and tried a simple experiment: write an interface with an input box and content that changes as the input box enters. This allows us to see what capabilities these tools provide and whether they actually help our programming efforts.

The sample code is as follows:

I fully agree that AI can help at all stages of software development, not just at the code level.

VS Code inspired

VS Code is currently the most popular editor for front-end developers, and it provides smart features by default that ordinary developers can quickly pick up and play with. We start with it and introduce some concepts and terms in the editor field.

IntelliSense

IntelliSense is a set of features built into VS Code, including Code Completion, Quick Info, and Parameter Info.

By default, VS Code provides smart tips for JavaScript/TypeScript, HTML, CSS/SCSS/Less, etc. You can also add smart tips for more languages by installing language plug-ins.

Code Completion

VS Code Smart hints are supported by VS Code’s language services. Language services provide code completion capabilities based on language semantics and source code analysis. Specifically, if the language service knows that typing is possible, the suggested completion list will pop up after you type:

If you continue typing characters, the completion list (variables, methods, and so on) is filtered to contain only lists containing typed characters:

VS Code Smart Tips provides a rich variety of completion tips, including suggestions for language services, snippets, and simple word-based text completion.

Within the completion list, their Sorting of Suggestions is as follows:

• Suggestions for language services

• Code snippet

• Global identifier

• Simple word-based text

Sort in alphabetical order within an item

A few special notes:

Proximity Principle (reference)

In the image above, you can see that count, context, and colocated are sorted by the range in which they appear (loop, function, file).

Nearby principle of contextual cues default is not open, you can use the editor, suggest. LocalityBonus set to open it.

Priority of snippets (reference)

. You can use the editor snippetSuggestions setup code fragment in code completion list is displayed and priority. To remove code snippets from the code completion list, set the value to “None”. If you want to view snippets, you can specify the order in which they are placed on the code completion list: at the top (” top “), at the bottom (” bottom “), or inline alphabetically (” inline “). The default value is inline.

Suggested completion items for code completion list (reference)

By default, VS Code pre-selects previously used items in the Code completion list. SuggestSelection you can use the editor.suggestSelection setting if you want to behave differently, for example, by always selecting the first item in a suggestion list.

Available values include:

• First: Always select the top list item.

• RecentlyUsed :(default) the previously used item is selected unless the prefix (the type to be selected) selects another item.

• RecentlyUsedByPrefix: Select items according to the prefixes suggested by previous completions.

Overview of list items (Quick Info)

On the code completion list, you can view the overview of each completion item by clicking the icon to the right of the list item, and the accompanying documentation for that completion item is expanded to the side. The expanded document stays the same and updates as you browse the list. You can close this window by clicking the “Close” icon:

Function signature Info

When you select a function (method) from the code completion list, information about the function signature is displayed. When there are more than one parameter, highlight (underline highlight) the parameter currently being completed:

Smart Actions

In addition to hints, VS Code also provides some smart coding tools that developers can take initiative with.

Refactoring

Code refactoring provides functions such as extracting functions and extracting variables. In VS Code editing, just select the source you want to extract and click the light in the slot or press (⌘) to see the refactoring available:

The ability to refactor code is provided by language services, so different languages can do different code refactorings. VS Code has built-in support for TypeScript and JavaScript Code refactoring through the TypeScript language service.

For more information about code refactoring, see the refactoring section of the TypeScript Language Service.

Quick Fixes

Quick Fixes are diagnostics provided by the language service that can be used to find common programming problems. For example, it can analyze your source code and detect code that will never be executed, which is grayed out in the editor. If you hover over such a source line, you can see a hover description, and if you place your cursor over the line, you will get a quick fix bulb:

Quick fixes vary from language to language, VS Code has some built-in quick fixes for TypeScript and JavaScript:

• Add the missing this to the member access

• Fix misspelled property names

• Remove unreachable code or unused imports

Developers can add more quick fixes by installing language plug-ins.

The experiment demonstrates

Without installing any external plug-ins, I tried to complete the Code in the sample with VS Code, which looked like this:

• Most of the features listed in the introduction section were triggered during the demo

• There are few types of code completion hints, including type completion of language keywords, name expressions and attribute expressions

• The correct rate of code completion is not high, and most of them are words and global variables

• I tried installing @types/ React, and the React API was much more efficient with language semantics

What are the products in the industry?

Visual Studio IntelliCode

Visual Studio IntelliCode Visual Studio IntelliCode (VS IntelliCode) is an official Microsoft Smart Code plug-in that was launched in July 2018 and released in August 2019. The latest update was at the end of June this year. IntelliCode supports multiple programming languages (JavaScript/C++/Python) and has been downloaded up to 6 million times in the VS Code plug-in market.

IntelliCode for VS Code

IntelliCode claims to provide intelligent-assisted development based on a combination of understanding of code context and machine learning. Specifically, the following functions are provided.

The first feature is smarter code completion. While common code completion is sorted alphabetically, VS IntelliCode extracts methods based on context and “understand your usage habits”, giving developers the most likely correct API to use. In the example shown below, you can see the IntelliCode promoted prediction API appear in a new section at the top of the list, with members prefixed by a star icon:

The same functionality can also be used when writing React component properties:

The second function is the intelligent completion of parameters. Specifically, when your input function is closed, you will be prompted for which variable to use as an argument to the function.

For example:

try { // do something... } catch (error) { // do something... console.error(); // When you type console.error(), the editor cursor will be in the error method // and a list of completions will appear, the first item being the error variable passed in the catch. }Copy the code

Unfortunately, this feature is not available in TypeScript and Javascript. Examples in C+ are listed here:

The third feature is code refactoring advice. The example given on the website is when we update multiple parts of code in the same way:

The plugin displays the update light bulb icon in the code slot, and clicking on the Update light bulb icon tells us what other code can be updated in the same way:

Click the list item to locate the code that can perform the proposed update, and apply the proposed change in the slot next to the code:

This feature may not seem very useful, since we can update all the code at once by using regular expression search and replace while refactoring. Perhaps the power is that you update the same code “unconsciously” in two places (it tracks your editing process and content locally and detects things that can be applied repeatedly), and then the plugin gives you a hint that you can update more similar code.

The experiment demonstrates

IntelliCode mainly enriches the completion of attribute expressions. In our experiment, the completion efficiency of IntelliCode is not significantly different from native VS Code.

Kite

Kite is an intelligent Code editing aid launched by a Silicon Valley startup in March 2017 that supports embedding as a plug-in across multiple ides. The VS Code plug-in was last updated in mid-July. Kite was originally only for Python, but now supports Javascript for some of its features. With over a million downloads in the VS Code plug-in market, it is one of the most popular VS Code plug-ins in the Python community.

Kite parent raised $17 million in series A funding in early 2019 and currently offers both A free version and A paid plan.

Kite for VS Code

Kite’s official website describes its smart code completion feature. Kite provides code completion that differs from typical code completion in several ways:

1. Provides completion for almost all JavaScript code, such as statements, functions, objects, and so on, such as after Spaces:
   
   

2. Kite provides full or multiple lines of code completion compared to just word completion:
   
   

3. Kite guesses which inputs you’re most likely to make at the moment, offering intelligent recommendations:
   
   

Using the VS Code plugin for JavaScript

In addition to code completion, Kite provides smart snippets, smart code search, document search, and more, but these are currently only available in Python.

The experiment demonstrates

• Kite’s code completion types are richer, and completion efficiency is significantly higher than the first two

• Kite doesn’t have a lot of options for code completion lists, I guess to improve accuracy. From the practical effect, triggered completion items are generally correct

• Kite’s Code completion response speed is not significantly different from VS Code’s native functionality

TabNine

The first version of TabNine was released at the end of 2018 with support for embedding as a plug-in in multiple ides, and the VS Code plug-in was last updated at the end of July. TabNine’s programming language has a very high coverage and supports most common programming languages (JavaScript/Java/Python…). , which has 300,000 downloads in the VS Code plugin market.

TabNine, which started out as open-source software for the community, was acquired earlier this year by Codota, which announced a $12 million funding round in April and now offers both a free version and a paid plan.

TabNice for VS Code

Going one step further than VS IntelliCode offering API auto-completion, TabNine offers a full range of code completion. As soon as you type in the editor, TabNine will give you auto-complete reminders. The TabNine website uses three examples to illustrate its capabilities. Respectively is:

1. Code that can be written based on annotation derivation, such as function names, parameters, and return value types that can be used based on a function’s annotation
   
   

2. Derive possible apis and passed parameters based on the context:
   
   

3. Understand usage patterns of common libraries (best practices) and recommend code completion based on context:
   
   

The experiment demonstrates

• TabNine implements full type completion, and you can see that every time you type TabNine, a code completion list appears indicating possible inputs

• Due to the high frequency of the hints, the accuracy rate is not as good as Kite, but TabNine pleasantly displays possible matches in percentages on the right panel without too much disruption to the user’s programming experience

• TabNine’s Code completion response speed is not significantly different from VS Code’s native functionality

aiXcoder

The first version of aiXcoder was released in mid-2018, followed by the VS Code plug-in at the end of 2019, which was last updated in early May. AiXcode supports many major ides (IntelliJ IDEA/Eclipse/VS Code) and major programming languages (JavaScript/Java/Python), featuring IntelliJ IDE and Java language. Probably due to its recent launch and the fact that the personal version is Java-only, the VS Code plugin market has seen only 3k+ downloads. Because it is a domestic software and the functional document is more comprehensive, so it is also in the research list.

The founding team of aiXcoder comes from Peking University laboratory, and the leader is Li Ge, an associate professor of Peking University. AiXcoder is a school-enterprise cooperation project, and its parent company, Silicon Heart Technology, has obtained angel investment of one million RMB in 18 years. AiXcoder is available in both personal and enterprise versions.

aiXcoder for VS Code

AiXCode’s official website provides a relatively complete and systematic introduction of its functions. According to its classification, the functions provided are as follows:

Intelligent code completion

• Single Token completion: that is, the completion of input such as variable name, object attribute and object method;

• Multi-token completion: Remember chain calls? Recommended completions for multiple chained apis appear if necessary, for example: document.getelementByid (‘foo’).style.top

• For (let I =0, j = foo.length; j >= i; i++)

• Multi-line completion: Complete multiple lines of code at one time
  
  

• Continuous multiple completion: After the user confirms the recommendation result of aiXcoder, aiXcoder immediately gives the following recommendation code

• Function parameter auto-completion: that is, when the user calls a function (method), it prompts the possible input of parameters
  
  

Code intelligent search

• Document search, enter keywords can search out the corresponding documents and examples;

• Similar code search: select a part of the code, search for similar code in Github, and click to replace; Search for codes with high similarity in function implementation in their own repository;

• Recommend code search, input the function description to be implemented (Chinese), recommend available code.

Quality and efficiency of programming intelligence

Through intelligent analysis, the quality and efficiency of the project can be obtained to help developers continue to improve.

The experiment demonstrates

Because the personal version of aiXcode only supports the Java language, we couldn’t experiment with our example.

A large company

Facbook

Facebook’s AI Lab launched Aroma in April 2019. Aroma is a code search and recommendation tool that uses machine learning techniques to make it easier and more efficient to get effective code from a large code base for programming assistance. By showing and developers are trying to write code similar to the sample (and assuming the samples corresponding to as part of the company code base of high quality code), the proposal can help developers to completion faster, can also help developers to detect possible errors or refactoring opportunities, such as lack of exception handling in the sample:

Google

I didn’t find Google’s tools for automatic or auxiliary ai programming in the public literature or news. But it turns out that Google has some research in this area. For example, a team layer at Google Brain published a paper called “Neural Networks for Modeling Source Code Editing,” in which they trained a network of millions of fine-grained edits from thousands of Python developers to predict future edits. In this paper, Google focuses less on static code and more on code editing as a dynamic object that evolves over time.

other

In addition to the tools above that provide a holistic intelligence solution, I’ve found some tools that focus on intelligentizing a single capability that may not be on a large scale, but still explore interesting directions:

• DeepCode: a plug-in that helps find code flaws, security holes, performance, and API usage issues in a project. The following illustration illustrates how DeepCode checks all problems using the “problems” TAB and syntax highlighting:
  
  

• Intelli Refactor: This plug-in is an enhancement to VS Code’s built-in Code refactoring capabilities.

Compare ability and experience

Finally, let’s make a horizontal comparison of the capability coverage and experience of these tools in the form of tables. The capability part is based on the introduction on the official website of the tools, and the experience part is based on my personal feelings in the experiment.

Because the experiment uses front-end code and is limited by the code design, my personal machine condition and programming habits, it cannot be used as a scientific basis and is only for reference.

Comparison of Main abilities

• Offline use: Does not depend on the network to use. Part of the software uses deep learning technology, so large computing resources are needed. Therefore, cloud resources are used and network environment is relied on.

• Domain customization: Use proprietary enterprise or team code to train a “dedicated intelligence engine” to make recommendations that are more consistent with enterprise or team programming habits.

Intelligent prompt comparison

• Completion type: that is, the situation that can trigger the completion prompt

• Attribute expression:object.${api}
  
  

• Name expression:
  
  

• Multi-token completion: as shown in the figure below,encodeIs a single Token completion,encode('utf8')That is, multi-token completion
  
  

• Full line completion: as shown in the figure below,postIs a single Token completion,post(my_url, data=my_data)It’s full row completion
  
  

• Multi-line completion: As shown below, multiple lines of code are completed at once
  
  

Comparison of User experience

• Completion efficiency: that is, how many times do you need to type the same code to complete the input? The less you type, the higher the completion efficiency is.

• Accuracy: that is, for the same code, the proportion between the correct times of the completion reminder preference and the total times of the completion reminder. The higher the proportion, the higher the accuracy.

• Corresponding speed: that is, how long after the user enters the completion prompt list, the average value of completion prompt can be tested through the same code;

• Hard disk resources: Offline use depends on machine learning model, so the model occupies hard disk resources.

Just a gimmick?

With this simple experiment, it’s fair to say that ai “automatic programming” has so far been more about marketing. I believe these tools will make a lot of progress in the next few years and could become true virtual assistants for developers, but we’re still some way off. At the same time, it is undeniable that the use of these tools has made developers more efficient in programming.

• For junior developers, smart programming tools can improve the efficiency and experience of programming, providing effective advice to help junior developers write better code faster;

• For advanced developers, smart programming tools are less productive, but they can still provide some fun programming experiences. Senior developers are skilled in the skills and knowledge of their fields. Programming is highly subjective, and they have their own taste in code. Code prompts can sometimes be a kind of interference, so intelligent programming tools should have the ability of personalized Settings.

• As with any method using machine learning, the effectiveness of an intelligent programming tool depends on the quality of the training data. Instead of using GitHub data to train the system, you will get better results by using internal/private repositories to train it.

In addition, at first, I only focused on the benefits of automatic programming for improving production efficiency, but in the process of research, I found that artificial intelligence automatic programming will also have a broad prospect for production quality and programming education.

In the future, I think we’ll also see programming robots that we can do some pair programming with and talk about what the goal of a particular approach is, and let the robot find the best solution for us.

Hard wide

Iceworks is a multi-application development suite that helps developers and r&d teams build multi-application faster and better. Welcome to Iceworks.

• AiXcoder head-on PK Kite!