1, an overview of the

From the introduction of the previous article, “Architectural Design: Inter-System Communication (10) — The Basic Concepts of RPC,” we believe that the reader has understood the basic concepts of RPC. To deepen this understanding, the next few articles will detail the implementation of Apache Thrift, a typical RPC specification. The introduction of Apache Thrift is divided into three articles. The first part explains the basic use of Apache Thrift. Part I will explain how Apache Thrift works (mainly focusing on the message format encapsulation used by Apache Thrift, the supported network IO model, and its client request handling). The next part analyzes the deficiencies of Apache Thrift and implements a self-designed RPC service governance management scheme based on Apache Thrift. This will help us to understand Dubbo’s service governance in the future.

2. Basic knowledge

Thrift was originally developed by Facebook as an RPC framework between languages within the system. Facebook contributed to the Apache Foundation in 2007 and entered the Apache Incubator, Apache Thrift, in May 2008. The main advantages of Apache Thrift compared to other RPC implementations are: Support for many languages (C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, etc.), high concurrency (remember from the previous article, The key points we mentioned that affect RPC performance? .

To support multiple languages, Apache Thrift has its own interface definition language and can generate code for a variety of programming languages through its code generator. This is a prerequisite to ensure communication between different languages. In order to implement a simple instance of Apache Thrift, we first need to explain the IDL of Apache Thrift.

2-1. Installation of the Thrift code generator program

If you are running under the environment of Windows Apache Thrift test, so you don’t need to install any tool, direct download Apache Thrift code generation under the Windows program www.apache.org/dyn/closer…. (As of this writing, version 0.9.3 of Apache Thrift is being used); If you are running under the Linux system, then download www.apache.org/dyn/closer…. And compile and install (a simple process that won’t be described here). Remember to add the run location to the environment variable after installation.

2-2. Outline of IDL format

Here is a simple IDL file definition:

Struct Request {1: Required string paramJSON; struct Request {1: Required string paramJSON; 2:required string serviceName; Struct Reponse {1:required RESCODE responeCode; 2:required string responseJSON; } # exception ServiceException {1: Required EXCCODE exceptionCode; 2:required string exceptionMess; } # enum RESCODE {_200=200; _500 = 500; _400 = 400; } # another enumeration enum EXCCODE {PARAMNOTFOUND = 2001; SERVICENOTFOUND = 2002; {HelloWorldService {Reponse send(1:Request Request) throws (1:ServiceException e); } 1234567891011121314151617181920212223242526272829303132333435363738Copy the code

The above IDL files can be directly used to generate code for various languages. Here are some common code generation commands for different languages:

Thrift # generate c++ thrift-0.9.3 -gen CPP./demoHello. Thrift # generate PHP thrift-0.9.3 -gen php. / demohello. thrift # generates node.js thrift-0.9.3 -gen js:node./ demohello. thrift # generates c# thrift-0.9.3 -gen csharp . / demoHello. Thrift # you can through the following command to view the generation command format thrift - 0.9.3 - help1234567891011121314151617Copy the code

2-2-1. Basic type

A basic type is a representation of the data that is supported in any language. Apache Thrift supports the following basic types:

  • Bool: True or false, one byte
  • Byte: indicates the signed byte
  • I16:16-bit signed integer
  • I32:32-bit signed integer
  • I64:64-bit signed integer
  • Double: 64-bit floating point type
  • String: string/character array
  • Binary: binary data (represented in Java as java.nio.bytebuffer)

2-2-2, struct structure

In object-oriented languages, it is “class definition”; In weakly typed and dynamic languages, it is “structure/structure”. Define the format as follows:

Struct < structure name > {< number > : nature [field] < field type > < field name > / = > < default value [; |,]} 1234Copy the code

Example:

struct Request {
    1:required binary paramJSON;
    2:required string serviceName
    3:optional i32 field1 = 0;
    4:optional i64 field2,
    5: list<map<string , string>> fields3
}1234567
Copy the code

  • Structure name: You can give different (case sensitive) names based on your business requirements. Note, however, that structure names in a set of IDL definition files cannot be repeated, and keywords (such as required, struct, and so on) that IDL already occupies cannot be used.

  • Serial numbers: Serial numbers are very important. Positive integers are used in order. This property is used when Apache Thrift is serialized.

  • Field properties: There are two types of keywords: Required and optional. If you do not specify these keywords, the system defaults to Required. Required means that the field must have a value, and Apache Thrift will serialize this field whenever it serializes; Optional indicates that this field does not necessarily have a value, and Apache Thrift will only serialize this field if it has a value.

  • Field types: In structs, the field type can be one of the base types, one of the previously defined structs, one of the Apache Thrift supported containers (set, map, list), or a defined enumeration. The type of the field must be specified.

  • Field name: Field names are case sensitive, cannot be duplicated, and cannot use keywords (such as required, struct, etc.) that IDL already uses.

  • Default: You can specify a default value for a field (or not).

  • Terminator: In structs, two terminators are supported. You can use “;” Or “, “. Of course you don’t have to use the terminator (the Apache Thrift code generator will recognize that on its own)

2-2-3, containers Collection/containers

Apache Thrift supports three types of containers, which are common in a variety of programming languages:

  • List < T > : An ordered list (ArrayList in JAVA). T can be some base type, some previously defined struct, some Apache Thrift supported container (set, map, list), or a defined enumeration. Elements in an ordered list allow repetition.

  • Set < T > : an unordered collection of elements (HashSet in JAVA). T can be some base type, some previously defined struct, some Apache Thrift supported container (set, map, list), or a defined enumeration. Elements in an unordered set of elements are not allowed to repeat, and once repeated, the latter element overwrites the previous one.

  • map

2-2-4, enmu enumeration

Enum < the enumeration name > {< enumeration field name > = > < enumerated values [; |,]} 123Copy the code

The following is an example:

enum RESCODE { _200=200; _500 = 500; _400 = 400; } 12345Copy the code

22-2-5. Definition of Constants

Apache Thrift allows you to define constants. Constants have a “const” keyword and can be of the base type of Apache Thrift, a previously defined struct, one of the containers supported by Apache Thrift (set, map, list), or a defined enumeration. The following is an example:

const i32 MY_INT_CONST = 111111; const i64 MY_LONG_CONST = 11111122222222333333334444444; const RESCODE MY_RESCODE = RESCODE._200; 12345Copy the code

2-2-6. Exception

Apache Thrift exception, used primarily when defining service interfaces. The struct keyword is defined like the exception keyword, as shown in the following example:

exception ServiceException { 1:required EXCCODE exceptionCode; 2:required string exceptionMess; } 1234Copy the code

2-2-7. Service Service interface

One of the most important IDL definitions in Apache Thrift. In subsequent code generation phases, these services defined by IDL constitute the basic remote process by which The Apache Thrift client calls the Apache Thrift server. Service The definition of the service interface is as follows:

Service service < name > {< void | return refers to the type > < name > service method ([< enter the serial number > refs: [required | optional] < parameter types > < parameter name >... ) [throws ([abnormal serial number > < : [required | optional] < exception type > < abnormal parameter name >...]]]} 123Copy the code

  • Service name: The service name can be customized according to your business requirements. Note that the service name is case sensitive. There are only two restrictions on service names in IDL: you cannot reuse the same name, and you cannot use keywords (such as required, struct, etc.) that IDL already uses.

  • Return value type: If the calling method does not have a return type, the keyword “void” can be used; It can be a base type of Apache Thrift, a previously defined struct, a container (set, map, list) supported by Apache Thrift, or a defined enumeration.

  • Service method name: The service method name can be customized according to your business requirements and is case sensitive. In the same service, do not use the same service method name for multiple methods (be careful), do not use IDL already occupied keywords.

  • Service method parameters: “into the serial number > refs: [required | optional] < parameter types > < parameter name >. Note that similar to struct field definitions, you can specify required or optional; If this parameter is not specified, the system defaults to required. If there are multiple parameter names in a service method, the parameter names must not be duplicated.

  • Abnormal service method: throws ([abnormal serial number > < : [required | optional] < exception type > < parameter name >. Throws keyword is the starting point of the definition of abnormal service method. Behind the throws keyword, you can define one or more different exception type.

The following is an example of an Apache Thrift service definition:

service HelloWorldService { Reponse send(1:Request request) throws (1:ServiceException e); } 123Copy the code

2-2-8. Namespace Indicates the namespace

Apache Thrift supports different namespaces for different languages:

namespace java testThrift.iface

namespace php testThrift.iface

namespace cpp testThrift.iface12345
Copy the code

In the 2-2-9 s, comments,

Apache Thrift supports several styles of annotation. This is to accommodate developers with different language backgrounds:

/* * Annotation 1: **/ / annotation 2 # annotation 31234567Copy the code

2-2-10 include keyword

If you have more than one IDL definition file in your entire project (the name of the IDL definition file can be arbitrarily chosen). Then you can use the include keyword to include another IDL file in the IDL definition file A:

include "other.thrift"1
Copy the code

Please note that double quotation marks must be used and “; “is not used. Or “, “ending.

This is the basic SYNTAX of IDL. It is not possible to cover every syntax and every detail due to space, but the syntax points above are enough for you to edit a business-friendly, flexible IDL definition. If you need to know more detailed Thrift IDL syntax, you can refer to the official document about: thrift.apache.org/docs/idl

2-3. The simplest Thrift code

  • Define an implementation of the Thrift business interface helloWorldService.iface:

    package testThrift.impl;

    import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.apache.thrift.TException;

    import testThrift.iface.HelloWorldService.Iface; import testThrift.iface.RESCODE; import testThrift.iface.Reponse; import testThrift.iface.Request;

    / * *

    • We define a concrete implementation of the HelloWorldService.Iface interface.
    • Note that the parent interface, HelloWorldService.iface, is generated by thrift’s code generation tool
    • To run this code, import maven-log4j support. Otherwise modify the logger. info method
    • @author yinwenjie

    / public class HelloWorldServiceImpl implements Iface { /* * 日志 */ private static final Log LOGGER = LogFactory.getLog(HelloWorldServiceImpl.class);

    /** * In the interface definition, only one method needs to be implemented. < br > * HelloWorldServiceImpl. Send Request (Request) throws TException < br > * you can accept the client's understanding into the interface ways a Request object, After processing is complete, a Reponse object is returned to the client. Both the Request object and the Reponse object are structures defined by IDL, and the corresponding JAVA code is generated by the code generation tool. */ @override public Reponse send(Request Request) throws TException {/* * Perform specific service processing. * */ String json = request.getParamJSON(); String serviceName = request.getServiceName(); HelloWorldServiceImpl. LOGGER. The info (" get json: "+ json +"; Get serviceName: "+ serviceName); Reponse response = new Reponse(); response.setResponeCode(RESCODE._200); response.setResponseJSON("{\"user\":\"yinwenjie\"}"); return response; }Copy the code

    } 123456789101112131415161718192021222324252627282930313233343536373839404142434445

As you can see, the specific business and process in the above code is no different from normal business code. Even the implementation of this code doesn’t know it will be called by a client in the Apache Thrift framework.

  • Then we started writing the server-side code for Apache Thrift:

    package testThrift.man;

    import java.util.concurrent.Executors;

    import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.apache.log4j.BasicConfigurator; import org.apache.thrift.TProcessor; import org.apache.thrift.protocol.TBinaryProtocol; import org.apache.thrift.server.TThreadPoolServer; import org.apache.thrift.server.TThreadPoolServer.Args; import org.apache.thrift.transport.TServerSocket;

    import testThrift.iface.HelloWorldService; import testThrift.iface.HelloWorldService.Iface; import testThrift.impl.HelloWorldServiceImpl;

    public class HelloBoServerDemo {

    static { BasicConfigurator.configure(); } private static final Log LOGGER = logFactory.getLog (helloBoServerDemo.class); public static final int SERVER_PORT = 9111; Public void startServer () {try {HelloBoServerDemo. LOGGER. The info (" see this sentence means that thrift preparations for the service side..." ); / / service execution controller (as long as it is how to realize the operation scheduling services) TProcessor TProcessor = new HelloWorldService. Processor < Iface > (new HelloWorldServiceImpl()); TServerSocket serverTransport = new TServerSocket(helloBoServerDemo.server_port); // Thrift service based on the blocking synchronous IO model. // Set the IO network model for this server, set the message format encapsulation, set the thread pool parameter Args tArgs = new Args(serverTransport); tArgs.processor(tprocessor); tArgs.protocolFactory(new TBinaryProtocol.Factory()); tArgs.executorService(Executors.newFixedThreadPool(100)); // Start the thrift service TThreadPoolServer server = new TThreadPoolServer(tArgs); server.serve(); } catch (Exception e) { HelloBoServerDemo.LOGGER.error(e); } } /** * @param args */ public static void main(String[] args) { HelloBoServerDemo server = new HelloBoServerDemo(); server.startServer(); }Copy the code

    } 1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162

The above code has a few points to make:

  1. TBinaryProtocol: A binary description format unique to this class code, Apache Thrift. It is characterized by the use of less transmission per unit of data. Apache Thrift also supports a variety of data formats, such as the familiar JSON format. We’ll look at the data formats in Apache Thrift in more detail later.

  2. TArgs. ExecutorService () : ExecutorService is an asynchronous task scheduling service interface provided by the JAVA JDK 1.5+ java.util.concurrent package, and an implementation of ThreadPoolExecutor, the JAVA standard thread pool.

  3. Server.serve (), because of the synchronous blocking network IO model, the main thread of the application will remain blocked after this sentence. But if there is no error in the underlying network state, this thread will stay there.

Also, use Log4j for the code in the HelloWorldServiceImpl class. If Log4j is not present in your test project, use System.out instead.

  • Next we code the simplest Apache Thrift Client:

    package testThrift.client;

    import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.apache.log4j.BasicConfigurator; import org.apache.thrift.protocol.TBinaryProtocol; import org.apache.thrift.protocol.TProtocol; import org.apache.thrift.transport.TSocket;

    import testThrift.iface.HelloWorldService; import testThrift.iface.Reponse; import testThrift.iface.Request;

    / * *

    • Thrift Client, also based on the synchronous blocking model.
    • @author yinwenjie

    */ public class HelloClient {

    static {
    BasicConfigurator.configure();
}

/**
 * 日志
 */
private static final Log LOGGER = LogFactory.getLog(HelloClient.class);

public static final void main(String[] args) throws Exception {
    // 服务器所在的IP和端口
    TSocket transport = new TSocket("127.0.0.1", 9111);
    TProtocol protocol = new TBinaryProtocol(transport);

    // 准备调用参数
    Request request = new Request("{\"param\":\"field1\"}", "\\mySerivce\\queryService");
    HelloWorldService.Client client = new HelloWorldService.Client(protocol);

    // 准备传输
    transport.open();
    // 正式调用接口
    Reponse reponse = client.send(request);
    // 一定要记住关闭
    transport.close();

    HelloClient.LOGGER.info("response = " + reponse);
}
Copy the code

    } 1234567891011121314151617181920212223242526272829303132333435363738394041424344454647

  • The NETWORK I/O model used by the Thrift client must be consistent with that used by the Thrift server. That is, if the server uses blocking synchronous IO model, then the client must use blocking synchronous IO model.

  • The message encapsulation format used by the Thrift client must be the same as that used by the Thrift server. That is, if the server uses the message format TBinaryProtocol for binary streams, then the client must also use the message format TBinaryProtocol for binary streams.

  • The rest of the code is either defined by IDL and generated by Thrift’s code generation tools; Or it’s not important code, so there’s no need to post it to save space. Here’s how it works.

  • Server side running effect

  • After receiving the client request, the server takes out the thread from the thread pool to run

Notice how the server runs after receiving a client request: it pulls out a thread from a thread pool and runs the concrete implementation of the service interface. Now let’s go into the details of how Apache Thrift works.

(pick up below) source yinwj.blog.csdn.net/article/det…