What is The Language Server Protocol (LSP)?

First of all, according to official explained that Microsoft. Making. IO/language – se… :

Language Server Protocol (LSP) is a unified communication Protocol proposed by Microsoft in 2016. This scenario defines a set of protocols to use between an editor or IDE and a language server that provides language capabilities such as auto-complete, go to definition, find all references, and so on.

Students may not know much about Language Servers. For example, when we use online programming tools, do we also have code hints, code error diagnostics, etc.? Behind the scenes runs an instance of the language Server process (there are also developer tools that are coupled to the Language Server themselves, such as Eclipse), which is responsible for parsing the code file you currently have open.

Editor/IDE on market, nature provides to the user code editor (e.g., open the file, edit corpus and find references, open the workspace, etc.) and the editor’s response behavior (such as completion, code diagnosis, etc.) are actually the same, may not achieve on individual function, but not escape the above content. In other words, these functions can be abstracted into a series of “behavioral events.”

The purpose of LSP proposed by Microsoft is to make the previous editors (VSCode, Vim, Atom, Sublime…) Each has its own set of features and protocols implemented inside the editor. For each new editor, it is possible to write a corresponding Language Server for each Language supported in the editor. That is to say, assuming that there are N languages and M editors, the development cost and complexity of all editors for all languages is N * M.

Could there be an abstraction in the middle layer that separates the language’s “static analysis services” from the “editor/IDE”? Development cost and complexity can be reduced to linear N + M in the above scenario.

For example, each editor (client) is responsible for generating the behavior events in the standard when the user generates some common behavior (such as clicking to jump to definition), and then invokes the Interface methods of the Language Server in jSON-RPC form. Language Server, in turn, must implement all of the interfaces defined by the LSP specification (or at least the key parts of it).

The advantage of this is that for a programming language, an editor tool does not need to worry about how to do code analysis, but only how to initiate or respond to RPC events specified by LSPS on the interface. On the language server side, the same thing, just need to pay attention to the protocol itself events and respond & initiate events.

This idea of separating the middle layer is very common in P.S. for example, compilers are divided into a front end and a back end. The front end produces the intermediate language IR, and the back end is responsible for translating the intermediate language into CPU-specific instruction sets. Typical examples are JVM bytecode, LLVM IR, etc.

Also, because the editor and The Language Server are two processes, if the Language Server dies, the editor process itself will still exist and users won’t have to worry about losing code that hasn’t been modified yet.

Any downsides? Yes, all editors and Maintainers of Language Server on the market require time and effort to be compatible with the protocol, and the protocol itself requires the Server/client to respond to the new protocol behavior as its version changes. But overall, the advantages outweigh the disadvantages.

The working mechanism of LSP

First, you need to know that LSP is a duplex protocol.

It is not only the developer tool (client) that actively communicates with the Language Server (Server), but the Server may initiate RPC requests to the developer tool (such as the code diagnosis event textDocument/Diagnostics, which can only be actively sent from the Server to the client).

In the LSP specification definition document, each RPC event is marked with the possible initiator and whether the initiator needs to respond.

Here are two examples:

  1. For example, a request event initiated by a client that requires a return from the server (with a left-to-right and then turn arrow in parentheses of the subtitle) :

  1. Another example is a request event initiated by a server that requires the client to return (with a right-to-left and then turn arrow in parentheses of the subtitle) :

  1. There are also unilateral sends, which do not require a response (tool to server unilateral/server to tool unilateral) :

Let’s take Goto Type Definition Request as an example to visualize the whole process. This RPC request may be initiated from VSCode by the user right-clicking on the “Goto Type Definition” event:

VSCode sends the following message to the Language Server process as IPC (for example, the actual parameter structure is complicated) :

{" jsonrpc ":" 2.0 ", "id" : 24, "method" : "textDocument/typeDefinition", "params" : {" textDocument ": {" uri" : "file:///User/bytedance/java-hello/src/main/java/Main.java" }, "position": { "line": 3, "character": 13 }, // ... Other parameters},}Copy the code

The Language Server then gets the command and does the following:

  1. Which is the method called textDocument/typeDefinition is information analysis of a symbol type definition.
  2. According to the argument, the instruction’s source file is main.java line 3, character 13 — which, upon analysis, is the symbol foo.
  3. The Server finds the location of type foo for the symbol of foo. When found, return json-rpc via IPC as well:
{"jsonrpc": "2.0", // The id in the Request is 24, so the Response ID of the Server must also be 24. "id": 24, "result": {"uri": "file:///User/bytedance/java-hello/src/main/java/Main.java", "range": { "start": { "line": 7, "character": 25 }, "end": { "line": 7, "character": 28 } } }, }Copy the code

Only the client can make the current user’s edit cursor jump to the specified position based on the parameters in the return value.

Life cycle of the LSP

The example in the previous section is just one example of communication between Language Server and developer tools. Throughout the editing process, the Language Server/developer tools parties communicate continuously through various request bodies.

For specification purposes, interactions in the Language Server Protocol generally follow the following lifecycle.

After a user opens a project or code file, the developer tools need to start a Language Server child process and establish communication as appropriate. After the Language Server starts receiving messages, it typically starts with an initialization request from the client.

  1. Initialize the Initialize ()

Because Language Server starts, you do not know the status of the current editor. Therefore, the initialize instruction is always the first RPC request made by all lSP-compliant developer tools after they connect to an LSP-compliant Language Server. The Initialize directive has a complex structure that tells Language Server where the workspace is currently, what capabilities the client provides, and so on.

After the Server is initialized according to the configuration information in the editor tool request body, it responds to the InitializeResult structure as a result and tells the client what capabilities the Server currently has.

[Note: Most of the server/client capabilities in LSP are optional due to different editors’ different implementations: For example, some clients do not provide the codeLens function, and some servers do not provide the code completion function, etc. Both parties will inform each other whether they have these capabilities during the initialization phase, so as to avoid some invalid function requests later.

[Note 2: Client support for textDocument/didOpen, textDocument/didChange, and textDocument/didClose notifications is mandatory according to the LSP specification, and clients cannot opt out of supporting them.]

  1. Open a file (textDocument/didOpen)

Then, whenever a user on the developer tools side opens (or has opened before Language Server initializes) a file, the developer tools issue a textDocument/didOpen notification to Language Server, Notifies the Language Server that a file is open.

As defined in the protocol specification:

The document open notification is sent from the client to the server to signal newly opened text documents. The document’s content is now managed by the client and the server must not try to read the document’s content using the document’s Uri. Open in this sense means it is managed by the client. It doesn’t necessarily mean that its content is presented in an editor. An open notification must not be sent more than once without a corresponding close notification send before. This means open and close notification must be balanced and the max open count for a particular textDocument is one. Note that a server’s ability to fulfill requests is independent of whether a text document is open or closed.

A Document Open notification is sent from the client to the server to represent a newly opened text document. The content of the document is now managed by the client, and the language server must not attempt to read the content of the document using the Uri of the document. In this sense, “open” means it is “managed” by the client. This does not necessarily mean that the content will be displayed in the editor. A client cannot send open notifications more than once without a corresponding “close notification” being sent before it — that is, open and close notifications must match one by one, and the maximum open count for a particular textDocument is 1. Note that the server’s ability to satisfy the request is independent of whether the text document is open or closed.

For example, let’s open the main.go file under /workspace with VSCode:

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello World go!")
}
Copy the code

The textDocument/didOpen notification structure that will be sent is:

{"jsonrpc": "2.0", "method": "textDocument/didOpen", "params": {"textDocument": {"uri": "File:///workspace/main.go", "languageId" : "go", "version" : 2, / / the file content here as Language the content of the virtual file Server in the initial state "text" : "package main\n\nimport (\n\t"fmt"\n)\n\nfunc main() {\n fmt.Println("Hello World go!" )\n}" } } }Copy the code

The overall flow chart is as follows:

We noticed that Language Server tries to maintain a “virtual” file structure when it learns that the file is open, rather than reading the actual contents of the corresponding file in the file system. Subsequent operations such as saving files are written directly to the file system by the developer tools. Language Server is not responsible for synchronizing file contents.

After that, the user’s editing behavior is notified to the Language Server in the form of event notification. The Language Server responds by maintaining and adjusting the data structure of the virtual file objects described above based on the editing behavior.

Of course, don’t get me wrong, Language Server can still access the file system.

  1. Edit file (textDocument/didChange)

Editing files always happens after the open event.

According to LSP specifications, there are three update modes allowed by Language Server for editing operations: no update, full update, and incremental update. Most Language servers, however, are generally in incremental update mode, sending “diff” from edits rather than the whole updated content. For example, we add a new line “a” to our code:

package main import ( "fmt" ) func main() { fmt.Println("Hello World go!" ) + a }Copy the code

The client generates the following JSON-RPC request:

{"jsonrpc":"2.0", "method":"textDocument/didChange", "params": {"textDocument": {"uri": "File:///workspace/main.go", "version" : 37 / / the version number is used to confirm the order of change of}, "contentChanges" : [{" range ": {" start" : { "line":8, "character":4 }, "end": { "line": 8, "character": 4 } }, "rangeLength": 0, "text": "a" }] } }Copy the code

The server then updates the internal data structure based on the contents of the current change to determine whether certain “behaviors” (such as code diagnostics, etc.) are generated.

  1. Close the file (textDocument/didClose)

According to the specification, a closed file usually corresponds to a file object that has already been opened by the client. I won’t repeat it here.

The document close notification is sent from the client to the server when the document got closed in the client. The Document’s master now exists where the document’s Uri points to (e.g. if the document’s Uri is a file Uri the master now Exists on disk). As with the open notification the close notification is about managing the document’s content. Receiving a close notification doesn’t mean that the document was open in an editor before Requires a previous open notification to be sent. Note that a server’s ability to fulfill requests is independent of whether a text document is open or closed.

When a document is closed on the client, a document closure notification is sent from the client to the server. The main file of the document now exists where the URI of the document points to (for example, if the URI of the document is the file URI, the main file now exists on disk). As with opening notifications, closing notifications is about managing document content. Receiving a close notification does not mean that the document was previously opened in the editor. Closing a notification requires sending a previous open notification. Note that the server’s ability to satisfy requests is independent of whether the text document is open or closed.

Common Questions about LSP

  1. Doesn’t the language server access files in the file system?

No, it is still possible for Language Server to read files on the file system that have not been opened by the editor.

The protocol only states that textDocument/didOpen simply does not allow Language Server to open the contents of the URI file “already opened by the client”. However, Language Server is allowed to read other “open files” in the workspace and open file context.

For example, when importing other libraries for code completion, Language Server needs to access the file system to get index information.

  1. How is code diagnostics implemented?

In general, by establishing abstract syntax tree, grammar analysis is performed to check grammar errors.

Some plug-ins or code diagnostic tools, such as ESLint, can iterate through nodes in the syntactic AST to find more Lint warnings/errors.

  1. How is code completion implemented?

According to the LSP, the client initiates a request for code completion based on the event, and the triggering type is as follows:

  1. The user enters an identifier (the editor automatically executes this event in most cases) or hits Ctrl/Cmd + Space
  2. The user is entering a key character (such as “.”)
  3. The completion list is incomplete and needs to be triggered again

The server then determines how to complete the code based on the location of the current input cursor and the context of the file. The principle behind this piece is relatively complex, which can be described in a separate article.