An overview of the
Thrift is a framework for interoperable and scalable services for the development of extensible, cross-language services. It combines a powerful software stack with a code generation engine, To build on C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, node.js, Smalltalk, And OCaml and other programming languages seamlessly integrated, efficient services.
Thrift was originally developed by Facebook, opened source in April 2007, and entered the Apache incubator in May 2008. Thrift allows you to define data types and service interfaces (IDL) in a simple definition file. As input files, the compiler generates code to easily generate seamless cross-programming languages for RPC client and server communication.
It transmits data in binary format, which is smaller than serialization methods such as XML and JSON, and is more advantageous for high-concurrency, high-data volume, and multi-language environments. Thrift contains three main components: Protocol, Transport, and Server, where Protocol defines how messages are serialized, Transport defines how messages are communicated between the client and server, and server is used to receive serialized messages from Transport. Deserialize it from Protocol, invoke the user-defined message handler, serialize the message handler’s responses, and then write them back to Transport.
Official website: Thrift.apache.org
The basic concept
Architecture diagram
The top of the stack is the code generated from the Thrift definition file. Client and processor code generated by the Thrift service. These are represented by the brown boxes in the figure. The red box also generates code for sent data structures (except for built-in types). Protocols and transports are part of the Thrift runtime library. So using Thrift you can define services, and you are free to change the protocol and transport without having to regenerate the code. Thrift also includes a server infrastructure for binding protocols and transports together. There are blocking, non-blocking, single-threaded and multi-threaded servers available. The “bottom I/O” portion of the stack varies depending on the language being developed. For Java and Python network I/O, the Thrift library makes use of the built-in libraries, while the C ++ implementation uses its own custom implementation. Data type:
Basic types:
Bool: A Boolean value, true or false, corresponding to Java Boolean
Byte: an 8-bit signed integer, corresponding to Java byte
I16: a 16-bit signed integer corresponding to the Java short
I32: a 32-bit signed integer corresponding to Java int
I64:64-bit signed integer, corresponding to Java long
Double: a 64-bit floating-point number corresponding to a Java double
String: unknown encoded text or binary string, corresponding to Java string
Structure type:
12. struct: defines a common object, similar to a struct definition in C, which in Java is a JavaBean collection type:
List: corresponding to Java ArrayList set: corresponding to Java HashSet map: corresponding to Java HashMap exception types:
Exception: the corresponding Java Exception service type:
Service: service class
Data Transport layer
TSocket — Using blocking I/O for transmission is the most common pattern
TFramedTransport – Transfers by block size in a non-blocking manner, similar to NIO in Java. To use the TFramedTransport transport layer, the server must be modified to a non-blocking service type
TNonblockingTransport – Uses a non-blocking approach for building asynchronous client data transfer Protocol
Thrift allows users to select transmission protocols between the client and server. The transmission protocols are generally divided into text and binary. To save bandwidth and improve transmission efficiency, binary transmission protocols are mostly used. Text-based protocols are sometimes used, depending on the actual requirements in the project/product. Common protocols are as follows:
TBinaryProtocol: binary format. TCompactProtocol: efficient and dense binary compression format TJSONProtocol: JSON format TSimpleJSONProtocol: Provide JSON only write protocol, the generated file is easily parsed by scripting language Note: The client and server protocols must be consistent.
Server Type Server
TSimpleServer — Single-threaded server side uses standard blocking I/O, generally used for testing
TThreadPoolServer – The multi-threaded server side uses standard blocking I/O to pre-create a set of threads to process requests.
TNonblockingServer – Multi-threaded server side uses non-blocking I/O, server and client side need to specify TFramedTransport data transfer method.
THsHaServer – semi-synchronous and semi-asynchronous server model, which needs to be specified as: TFramedTransport Data transfer mode. It uses a separate thread to handle network I/O and a separate pool of worker threads to handle messages. This way, messages are processed immediately whenever there are free worker threads, so multiple messages can be processed in parallel.
TThreadedSelectorServer – TThreadedSelectorServer allows you to handle network I/O with multiple threads. It maintains two thread pools, one for network I/O and one for request processing. TThreadedSelectorServer performs better than THsHaServer when network I/O is the bottleneck.
Implementation logic
The service side
Implement the service processing interface IMPL
Create TProcessor Create TServerTransport Create TProtocol Create TServer Start the Server client
Create TProtocol. Create Client based on TTransport and TProtocol. Call the corresponding method ThriftServerDemo instance of Client
Create a new Maven project and add thrift dependencies
Namespace Java thrift. Service // Calculation type – Integer only four operations enum
ComputeType
{ ADD
0; SUB
1; MUL
2; DIV
3; } // service request struct
ComputeRequest
{
1 : required i64 x ;
2 : required i64 y ;
3 : required ComputeType computeType ; } // Service response struct
ComputeResponse
{
1 : required i32 errorNo ;
2 : optional string errorMsg ;
3 : required i64 computeRet ; } service ComputeServer
{
ComputeResponse getComputeResult ( 1 : ComputeRequest request ); } execute the compile command:
thrift
0.11 . 0.exe
r
gen java computeServer . thrift
Copy the generated Service class file to IDEA
Server interface implementation
public
class
ThriftTestImpl
implements
ComputeServer . Iface
{
private
static
final
Logger logger
LogManager . getLogger ( ThriftTestImpl . class );
public
ComputeResponse getComputeResult ( ComputeRequest request )
{
ComputeType computeType
request . getComputeType ();
long x
request . getX ();
long y
request . getY (); logger . info ( “get compute result begin. [x:{}] [y:{}] [type:{}]” , x , y , computeType . toString ());
long begin
System . currentTimeMillis ();
ComputeResponse response
new
ComputeResponse (); response . setErrorNo ( 0 );
try
{
long ret ;
if
( computeType
ComputeType . ADD )
{ ret
add ( x , y ); response . setComputeRet ( ret );
}
else
if
( computeType
ComputeType . SUB )
{ ret
sub ( x , y ); response . setComputeRet ( ret );
}
else
if
( computeType
ComputeType . MUL )
{ ret
mul ( x , y ); response . setComputeRet ( ret );
}
else
{ ret
div ( x , y ); response . setComputeRet ( ret );
}
}
catch
( Exception e )
{ response . setErrorNo ( 1001 ); response . setErrorMsg ( e . getMessage ()); logger . error ( “exception:” , e );
}
long end
System . currentTimeMillis (); logger . info ( “get compute result end. [errno:{}] cost:[{}ms]” , response . getErrorNo (),
( end
begin ));
return response ;
}
private
long add ( long x ,
long y )
{
return x + y ;
}
private
long sub ( long x ,
long y )
{
return x
y ;
}
private
long mul ( long x ,
long y )
{
return x * y ;
}
private
long div ( long x ,
long y )
{
return x / y ;
}} server implementation
public
class
ServerMain
{
private
static
final
Logger logger
LogManager . getLogger ( ServerMain . class );
public
static
void main ( String [] args )
{
try
{
// Implement the service processing interface IMPL
ThriftTestImpl workImpl
new
ThriftTestImpl ();
/ / create TProcessor
TProcessor tProcessor
new
ComputeServer . Processor < ComputeServer . Iface
(
workImpl );
// to create TServerTransport, non-blocking I/O, the server and client need to specify the TFramedTransport data transfer method
final
TNonblockingServerTransport transport
new
TNonblockingServerSocket ( 9999 );
/ / create a TProtocol
TThreadedSelectorServer . Args ttpsArgs
new
TThreadedSelectorServer . Args ( transport ); ttpsArgs . transportFactory ( new
TFramedTransport . Factory ());
// Deserialize the binary format ttpsargs.protocolFactory (new)
TBinaryProtocol . Factory ()); ttpsArgs . processor ( tProcessor ); ttpsArgs . selectorThreads ( 16 ); ttpsArgs . workerThreads ( 32 ); logger . info ( “compute service server on port :”
9999);
Create TServer / /
TServer server
new
TThreadedSelectorServer ( ttpsArgs );
// Start Server server.serve ();
}
catch
( Exception e )
{ logger . error ( e );
}
}} Server overall code structure
Log4j2.xml configuration file
<configuration
status
“INFO”
monitorInterval
“30”
<console
name
“Console”
target
“SYSTEM_OUT”
<PatternLayout
pattern
“%highlight{[ %p ] [%-d{yyyy-MM-dd HH:mm:ss}] [%l] %m%n}” />
<RollingFile
name
“RollingFileInfo”
fileName
“log/log.log”
filePattern
“log/log.log.%d{yyyy-MM-dd}”
<ThresholdFilter
level
“info”
onMatch
” ACCEPT “
onMismatch
” DENY ” />
<PatternLayout
pattern
“[ %p ] [%-d{yyyy-MM-dd HH:mm:ss}] [ LOGID:%X{logid} ] [%l] %m%n” />
<TimeBasedTriggeringPolicy
modulate
“true”
interval
“1” / >
<RollingFile
name
“RollingFileError”
fileName
“log/error.log”
filePattern
“log/error.log.%d{yyyy-MM-dd}”
<ThresholdFilter
level
“warn”
onMatch
” ACCEPT “
onMismatch
” DENY “
/>
<PatternLayout
pattern
“[ %p ] %-d{yyyy-MM-dd HH:mm:ss} [ %t:%r ] [%l] %m%n” />
<TimeBasedTriggeringPolicy
modulate
“true”
interval
“1” / >
<logger
name
“org.springframework”
level
“INFO”
<logger
name
“org.mybatis”
level
“INFO”
<root
level
“all”
<appender-ref
ref
“Console” />
<appender-ref
ref
“RollingFileInfo” />
<appender-ref
ref
“RollingFileError” />
Using JMeter to call the Java test class to call the corresponding background service, and remember each call and get the feedback value of RT,ERR%, only need to implement the test business in the way of single thread, there is no need to add various buried points to collect data
Create a new JavaMaven project and add JMeter and thrift dependencies
public
class
ThriftClient
{
private
ComputeServer . Client client
null ;
private
TTransport tTransport
null ;
public
ThriftClient ( String ip , int port ){
try
{
TTransport tTransport
new
TFramedTransport ( new
TSocket ( ip , port )); tTransport . open ();
TProtocol tProtocol
new
TBinaryProtocol ( tTransport ); client
new
ComputeServer . Client ( tProtocol );
}
catch
( TTransportException e )
{ e . printStackTrace ();
}
}
public
ComputeResponse getResponse ( ComputeRequest request ){
try
{
ComputeResponse response
client . getComputeResult ( request );
return response ;
}
catch
( TException e )
{ e . printStackTrace ();
return
null ;
}
}
public
void close (){
if
( tTransport ! =
null
&& tTransport . isOpen ()){ tTransport . close ();
}
}} Note: you need to copy the IDL interface file to the project
Create a JavaClass, TestThriftByJmeter in the example below, and AbstractJavaSamplerClient inheritance. AbstractJavaSamplerClient method to realize the four can override the default, respectively is getDefaultParameters (), setupTest (), runTest () and teardownTest () method.
The getDefaultParameters method is mainly used to set the parameters passed into the interface; The setupTest method is the initialization method used to initialize each thread during the performance test. The runTest method is the thread running body during the performance test. The teardownTest method is the test end method and is used to terminate each thread in a performance test. Write the TestThriftByJmeter test class
public
class
TestThriftByJmeter
extends
AbstractJavaSamplerClient
{
private
ThriftClient client ;
private
ComputeRequest request ;
private
ComputeResponse response ;
// Set the parameters passed to the interface
@Override
public
Arguments getDefaultParameters (){
Arguments arguments
new
Arguments (); Arguments. addArgument (” IP “, “172.16.14.251”); arguments . addArgument ( “port” , “9999” ); arguments . addArgument ( “X” , “0” ); arguments . addArgument ( “Y” , “0” ); arguments . addArgument ( “type” , “0” );
return arguments ;
}
// Initialize method
@Override
public
void setupTest ( JavaSamplerContext context ){
// Get the parameters set in Jmeter
String ip
context . getParameter ( “ip” );
int port
context . getIntParameter ( “port” );
int x
context . getIntParameter ( “X” );
int y
context . getIntParameter ( “Y” );
ComputeType type
ComputeType . findByValue ( context . getIntParameter ( “type” ));
// Create a client client
new
ThriftClient ( ip , port );
// Set request request request
new
ComputeRequest ( x , y , type );
super . setupTest ( context );
}
// Performance test thread runtime body
@Override
public
SampleResult runTest ( JavaSamplerContext context )
{
SampleResult result
new
SampleResult ();
Result.samplestart ();
try
{
long begin
System . currentTimeMillis (); response
client . getResponse ( request );
long cost
( System . currentTimeMillis ()
begin );
// Prints the timestamp difference. Java request response time
System.out.println (response.toString ()+ “total cost: [” + cost + “ms]”);
if
( response
null ){
// Set the test result to fasle result.setsuccessful (false);
return result ;
}
if
( response . getErrorNo ()
= =
0 ){
// Set the test result to true result. setSuccessful (true);
} else { result . setSuccessful ( false ); result . setResponseMessage ( “ERROR” );
}
} catch
( Exception e ){ result . setSuccessful ( false ); result . setResponseMessage ( “ERROR” ); e . printStackTrace ();
} finally
{
Result.sampleend ();
}
return result ;
}
// Test end method
public
void tearDownTest ( JavaSamplerContext context )
{
if
( client ! =
null )
{ client . close ();
}
super . teardownTest ( context );
}} Special note:
result . setSamplerLabel ( “7D” );
// Set the Java Sampler’s title result.setresponseok ();
Result.setresponsedata ();
// Write the test Run Main method to set the response content
public
class
RunMain
{
public
static
void main ( String [] args )
{
Arguments arguments
new
Arguments (); Arguments. addArgument (” IP “, “172.16.14.251”); arguments . addArgument ( “port” , “9999” ); arguments . addArgument ( “X” , “1” ); arguments . addArgument ( “Y” , “3” ); arguments . addArgument ( “type” , “0” );
JavaSamplerContext context
new
JavaSamplerContext ( arguments );
TestThriftByJmeter jmeter
new
TestThriftByJmeter (); jmeter . setupTest ( context ); jmeter . runTest ( context ); jmeter . tearDownTest ( context );
}} Test result passed
If your jar depends on another third-party jar, you need to place it in lib/ext, otherwise you will get a ClassNotFound error. If you can’t find your class after placing the jar in lib/ext, and you have JMeter on, You need to restart JMeter to open JMeter and select the JMeter test class when adding Java requests. You can see the parameters and default values in the list below.
