Code base for this blog:

HttpUtil2

Description:

1. Design the front-end and back-end interaction protocols

The convention is a data-code-MSG three-field design

There is also data-code-msg-isSuccess, where isSuccess and code are actually redundant.

However, after looking at the interface interaction protocols of Facebook, Google and other large companies, I found that the most comprehensive ones are:

data-code-msg-errorData.

When the request is correct:

{ "data": { "uid": "898997899788997" }, "code": "0", "msg": "success!" , "success": true, "errorData": null }Copy the code

Request error

ErrorData should be parsed using map to avoid parsing exceptions. Or just use optJSONObject(“errorData”)

{
  "data": null,
  "code": "user.login.401",
  "msg": "unlogin",
  "success": false,
  "errorData": {
    "reason":"kickout",
    "time":1689799989
  }
}
Copy the code

For debugging convenience, in the development/test environment, when the background is 500, the exception stack information should be directly inserted in MSG and returned to the front end.

2. What features should be included

The underlying

From urLConnection to httpClient to okHTTP

Encapsulation layer

From volley/asyncHttpClient to retrofit

Today, it’s basically OKHTTP with the bottom layer and Retrofit + RxJava with the top layer.

Even with Retrofit, there is still a lot of repetitive code to write, which requires a further layer of encapsulation to facilitate daily CRTL + C, CTRL + V.

Even CRTL + C would like a little less code.

So what functions do you need for a well-packaged network framework?

Let’s take a look at some of star’s more encapsulated libraries:

OkGo:

The NET of the Large Hadron

rxHttp

Based on daily development experience, there are actually the following things that can be tucked into the framework:

In fact, on second thought, a complete client-side network library should be configurable and simple-to-use like Postman.

Encapsulating a network framework is nothing more than a GUI turned into an API.

3. Several design ideas

Full information is accessible

The full information about the request and response should be available in the callback.

For example, okHTTP can retrieve the entire call object and the entire response information in its callback.

Many framework callbacks only have parsed data in them. When you need to use other information, you’re ata loss.

Fully fit the page life cycle

Tube you preach the view, fragments, activity, lifecycleowner, viewmodel, all automatic processing.

How do you say the view gets the page lifecycle? If you peel off the context, you always get the activity.

The request is automatically cancelled after the lifecycle ends.

There are two ways to cancel a request:

(There is no difference in waiting queue, both are removed from queue –> only… Okhttp-rxjava threading model, basically all immediately issued, no wait)

Direct socket.close() to close the connection
Does not interfere with the underlying level, but cuts the callback through the Boolean value in the callback

Both Retrofit and RxJava takeutil are the first. Simple and easy to implement, but the back-end interface monitoring has some more 0 or 499 errors.

No kotlin coroutines

Kotlin coroutines are awesome, right? Sorry, just a fake coroutine, the underlying thread pool switch. Just write asynchronous code synchronously (similar to js async,await).

It’s not that Kotlin isn’t good, but that the JVM itself doesn’t support coroutines.

To actually implement true coroutines like GO, or to jump out of the JVM and call Epoll for multiplexing myself, would be awesome, and I’d be jumping to kotlin to rewrite the framework.

Let’s start with the implementation and use of each key point

4. API usage:

Direct look at the readme

HttpUtil.requestAsJsonArray("article/getArticleCommentList/v1.json",PostStandardJsonArray.class) .addParam("pageSize","30") .addParam("articleId","1738") .addParam("pageIndex","1") .post() .setCacheMode(CacheMode.FIRST_CACHE_THEN_REQUEST) // .setCacheMode(CacheStrategy.REQUEST_FAILED_READ_CACHE) .callback(new MyNetCallback<ResponseBean<List<PostStandardJsonArray>>>(true,null) { @Override public void onSuccess(ResponseBean<List<PostStandardJsonArray>> response) { MyLog.json(MyJson.toJsonStr(response.data)); } @Override public void onError(String msgCanShow) { MyLog.e(msgCanShow); }});Copy the code

String url2 = "https://kiwivm.64clouds.com/dist/openvpn-install-2.4.5-I601.exe"; HttpUtil.download(url2) .setFileDownlodConfig( FileDownlodConfig.newBuilder() .verifyBySha1("76DAB206AE43FB81A15E9E54CAC87EA94BB5B384") .isOpenAfterSuccess(true) .build()) .callback(new MyNetCallback<ResponseBean<FileDownlodConfig>>() { @Override public void onSuccess(ResponseBean<FileDownlodConfig> response) { MyLog.i("path:"+response.data.filePath); } @Override public void onError(String msgCanShow) { MyLog.e(msgCanShow); }});Copy the code

5. The key point

5.1 Synchronous and asynchronous support

Okhttp itself has both synchronous and asynchronous writing.

Synchronous direct return, wrapped with try-catch.

For asynchrony, a callback is used.

But we use Retrofit internally here, based on RxJava. It all becomes a callback.

Instead of writing synchronously, write asynchronously.

How does RxJava synchronize?

There is no thread switch, and it is executed synchronously

HttpUtil. RequestString (" article/getArticleCommentList/v1. Json "), post (). The setSync (true) / / synchronization execution. The addParam (" pageSize ", "30") .addParam("articleId","1738") .addParam("pageIndex","1") .callback(new MyNetCallback<ResponseBean<String>>(true,null) { @Override public void onSuccess(ResponseBean<String> response) { MyLog.i(response.data); } @Override public void onError(String msgCanShow) { MyLog.e(msgCanShow); }});Copy the code

5.2 Automatically Processing the Life cycle

Primitive times:

The way this library is used.

Static map is used to store activity/fragment objects and requests. When the activity/fragment destory is performed, the request is retrieved from the map, the status is determined, and the request is cancelled.

/** * Cancel the request, usually called in the activity ondeStory. Public void cancelByTag(Object obj) {if (obj == null) {if (obj == null) { return; } List<retrofit2.Call> calls = callMap.remove(obj); // Delete from the gc root reference if (calls! = null && calls.size() > 0) { for (retrofit2.Call call : calls) { try { if (call.isCanceled()) { return; } call.cancel(); } catch (Exception e) { ExceptionReporterHelper.reportException(e); }}}}Copy the code

RxLifecycle + rxjava

OnDestory builds Transformer and passes it to RxJava’s takeUtil operator.

This library is not implemented

livedata

Observable turns to LiveData and hooks directly to lifecyclerOwner.

This library has been implemented.

5.3 General UI support

loadingDialog

Built-in, not displayed by default. Configurable switches,UI styles

Wrong toast for MSG

A more convenient way to do this is to do it uniformly in onError, which is turned off by default and can be turned on via the chained API.

The test environment should toast: code+”\n”+ MSG. And the MSG of the test environment should contain the stack information as far as possible.

Error code transfer to copy

Generally, it should be handled uniformly within the framework.

There are three types:

Exceptions thrown by the underlying framework should be turned into friendly copywriting

The error codes of the HTTP request itself, such as 400,500, should be uniformly copywritten

In the business data-code-MSG, if the MSG part of the background does not do internationalization, then the client should configure the corresponding translation copy.

The framework should automatically handle the first two and provide a configuration interface for the third type of business error copywriting:

ExceptionFriendlyMsg.init(context, new IFriendlyMsg() { Map<String,Integer> errorMsgs = new HashMap<>(); { errorMsgs.put("user.login.89899",R.string.httputl_unlogin_error); } @Override public String toMsg(String code) { Integer res = errorMsgs.get(code); if(res ! = null && res ! = 0){ return context.getResources().getString(res); } return ""; }});Copy the code

Internal copywriting has been configured :(Chinese + English)

5.4 Response physical check

The Bean Validator is often used when the server receives client/browser requests. Has evolved into a Java specification.

The need is not strong on the client side. The return of the server is more stable in most cases, and the loss of fields, field errors and other conditions are less.

However, in order to install a small X, I still implement this function –>

In fact, it is not the implementation, but the common functions of the server end are migrated to the mobile end and adapted. Did a little bit of work.

Please look at:

AndroidBeanValidator

To port to Android, you need to consider java8 compatibility issues, performance (method time), and impact on apk size. The default is Apache BVal 1.1.2.

String errorMsg = BeanValidator.validate(bean); // If the errorMsg returned is null, the validation passed if(! TextUtils.isEmpty(errorMsg)){ //Toast.makeText(this,errorMsg,Toast.LENGTH_LONG).show(); Observable. Error (XXX)// Throw out the errorMsg and the specified errorCode}else {// Get the proper bean}Copy the code

Do this when the bean is parsed.

5.5 Cache control: Rich cache patterns

Caching control patterns beyond the HTTP protocol itself

What are the limitations of HTTP cache control itself:

Only GET requests can be cached
Old complex request header

Write your own client, can accept this gas? It’s got to change, big change!

Be able to cache any request
Be able to support common business modes with one click

.setCacheMode(CacheMode.FIRST_CACHE_THEN_REQUEST)
Copy the code

/ / caching strategies, classification reference: HTTP: / / https://github.com/jeasonlzy/okhttp-OkGo / / don't use the cache, this mode, cacheKey, cacheMaxAge parameters invalid public static final ints NO_CACHE = 1; // Follow the default caching rules of the HTTP protocol, such as when there are 304 response headers. public static final int DEFAULT = 2; // First request the network. If the request fails, the cache is read. If the cache fails to be read, the request fails. Public static final int REQUEST_FAILED_READ_CACHE = 3; Public static final int IF_NONE_CACHE_REQUEST = 4; public static final int IF_NONE_CACHE_REQUEST = 4; // Use the cache first, whether it exists or not, Public static final int FIRST_CACHE_THEN_REQUEST = 5; public static final int FIRST_CACHE_THEN_REQUEST = 5; Public static final int ONLY_CACHE = 6;Copy the code

5.6 the cookie

Okhttp has no underlying cookie by default, but it does provide an interface that we implement based on its interface, the CookieJar.

Generally speaking, there are:

Do not store cookies
Cookies are stored only in memory
Cookie serialized to sharePrefences/file:

The third is more like browser behavior. It just doesn’t have all the cross-domain, security restrictions that browsers have to disgusting to play with.

HttpOnly you say? sameSite? It doesn’t exist. It’s just a bunch of key-values. You can do whatever you want.

However, as a framework, it is necessary to follow the basic principle of responding to host and path. Otherwise, provide interfaces for others to customize. Loose or strict as you like.

public static final int COOKIE_NONE = 1; public static final int COOKIE_MEMORY = 2; public static final int COOKIE_DISK = 3; private int cookieMode = COOKIE_DISK; */ public GlobalConfig setCookieMode(int cookieMode) {this.cookiemode = cookieMode; return this; }Copy the code

5.7 Common request header, request parameters/request body parameters

Map can be initialized and added on each request:

This is recommended if the value does not change after initialization.

If it changes, you can’t use this. Or update the cache map after the change.

You can also add interceptors using okHTTP interceptors.

For the request header, the GET request, it’s very simple to implement

But for post JSON or multiPart, you need to change json back to map, and then add, and then restore multiPart, and then add.

May refer to:

AddCommonHeaderAndParamInterceptor

If the request body signature is involved, be sure to add this interceptor before the signature interceptor.

5.8 Request Timeout

Does okHTTP have a timeout?

Before, there were only connecTimeout,read, and write timeouts. Now, callTimeout has been added to cover the entire request process at the OKHTTP level.

In the days when there was no callTimeout, the following three Settings were not sufficient, because the DNS resolution process could not be overridden by them.

You can use rxJava’s timeout to control the duration of the entire process.

Today, rxJava is still preferred for control. Because the callTimeout of OKHTTP cannot override the custom cache read/write timeout.

This typically provides a global configuration and a single request configuration

5.9 Request Retry

Okhttp itself has a retry API:

builder.retryOnConnectionFailure(boolean)
Copy the code

But only TCP connection failed retry. You can retry only once

To retry any error, and to specify the number of retries, you need to use the RxJava API. I don’t need to talk about it, just use it.

5.10 Exception catching and Reporting

Regardless of whether okHTTP/Retrofit crashes or not, you, as a wrapper framework, must not crash.

In any case, it must be 100% crash free.

There are several key points:

Within the interceptor

As the first application interceptor, add a try-catch to chain-.proceed (request) and demote it to an ioException, which can be handled by okHTTP’s error callback.

@Override public Response intercept(Chain chain) throws IOException { try { Response response = chain.proceed(chain.request()); } catch (Throwable e) { if (e instanceof IOException) { throw e; } else {// degrade so that the okHTTP framework can handle errors instead of crash throw new IOException(e); }}}Copy the code

Rxjava global exception capture:

This is usually done in the main project. No participation within the framework.

RxJavaPlugins.setErrorHandler(new Consumer<Throwable>() { @Override public void accept(Throwable e) throws Exception { report(e); }});Copy the code

Its own framework layer callback

The onSuccess and onError of the callback are consumer implementations. What if a crash also occurs? Get it for you, too!

OnSuccess throws an exception, degraded to onError

OnError also throws an exception, mimics RxJava, and degrades to global error handling

if(bean.success){ try { onSuccess(callback,t); }catch (Throwable throwable){ onError(callback,throwable); } }else { onError(callback,bean.errorInfo); } private static <T> void onError(MyNetCallback<T> callback, Throwable e) { try { Tool.logd("-->http is onError: "+callback.getUrl() ); Tool.dismissLoadingDialog(callback.dialogConfig, callback.tagForCancel); ErrorCallbackDispatcher.dispatchException(callback, e); }catch (Throwable e2){ if(GlobalConfig.get().getErrorHandler() ! = null){ try { GlobalConfig.get().getErrorHandler().accept(e2); } catch (Exception exception) { exception.printStackTrace(); } }else { if(! GlobalConfig.get().isDebug()){ e2.printStackTrace(); If (globalconfig.get ().isdebug ()){throw e2; }}}Copy the code

5.11 the debug function

The main form of network debugging is packet capture

Provide a variety of forms to see the bag:

logcat

Transformation okhttpLoggingInterceptor, request body response body line of print directly, convenient to copy. More than 4000 characters cut into branches.
Grab a bag inside the phone

Modified Chuck, based on okHTTP interceptor, notification bar shows captured packet content. Provides filtering for excessive surge requests, such as various behavior log reports, etc.

The PC agent captures packets

Usually fiddler or Chales.

The following configuration is required: The debugable environment later than 7.0 ignores certificates

<network-security-config>
    <debug-overrides>
        <trust-anchors>
            <certificates src="system"/>
            <certificates src="user"/>
        </trust-anchors>
    </debug-overrides>
    <base-config cleartextTrafficPermitted="true">
        <trust-anchors>
            <certificates src="system" />
        </trust-anchors>
    </base-config>
</network-security-config>
Copy the code

Or directly the network framework ignores the certificate in the debug environment

stetho-> flipper

Based on the OKHTTP interceptor, the captured packet content is sent to the client on the PC for display. The display interface is more high-end and classy.

Rewrite flipper’s built-in interceptor, with extra encryption, decrypted and sent in plain text.

One line script integration: flipperUtil

5.12 Online monitoring

Reporting is no trouble, the key is how to do statistical analysis? What’s available, how to build your own.

Add a report in the above interceptor. The key is where to report it

Build the exception and report it to Sentry.

Or build your own Flume + ELK analysis system.

Or a little creepy, build the event to report to the event statistics platform, traffic traffic.

Which parameters

Error message:

Get it in the above interceptor/uniform error callback and report it. Generally, the fault is reported to the statistics platform to check the error trend. Based on the trend, you can check whether the front and background services are abnormal in a certain period of time. This is usually just a supplement to the monitoring requested by the background itself.

A few years ago, I used the real-time event analysis function of Google Analytics to turn error messages into event reports. I could see the network error trend within 1min and 30min in real time, with its own sorting, which was very cool. Unfortunately, Google Analytics went offline later, and this function on FireBase was occupied by operation.
Request time sharing information:

Such as DNS time, TCP time, TLS, HTTP request response, these can be obtained through okHTTP eventListener interface.

5.13 security

The basic means are:

Some of the operations you play on HTTPS
Custom encryption
Request header, request body signature – tamper-proof

https

These are basically the questions

What is a man-in-the-middle attack

How to prevent man-in-the-middle attacks

What is one-way certificate verification and how does the framework layer implement it

What is bidirectional certificate verification and how does the framework layer implement it

How to combat certificate verification? Root phone + Frida + OkHttplogging dex reference: Frida use

Custom encryption

Get the requestBody byte array in the interceptor, encrypt it, build a new requestBody, and move on.

final Buffer buffer = new Buffer(); requestBody.writeTo(buffer); final long size = buffer.size(); final byte[] bytes = new byte[(int) size]; buffer.readFully(bytes); final byte[] bytesEncrypted = encrypt(bytes); Return RequestBody() {@override public MediaType contentType() {return RequestBody() {return public MediaType contentType() MediaType.parse(type); } @Override public long contentLength() { return bytesEncrypted.length; } @Override public void writeTo(BufferedSink sink) throws IOException { sink.write(bytesEncrypted); }};Copy the code

Request header request body signature

It’s just adding salt to produce SHA1, SHA256 and so on, nothing to talk about.

5.14 gzip

Okhttp already has built-in GZIP processing for the response body, so needless to say.

If the request body is a large string, then gZIP compression is ok in terms of traffic revenue.

Requires front and rear end support.

We do gZIP compression in the interceptor.

You cannot specify the size after gzip. You can wrap a layer to set the contentLength of the request body

private RequestBody gzip(final RequestBody body, String type) { return new RequestBody() { @Override public MediaType contentType() { return body.contentType(); } @Override public long contentLength() { return -1; // We don't know the compressed length in advance! } @Override public void writeTo(BufferedSink sink) throws IOException { BufferedSink gzipSink = Okio.buffer(new GzipSink(sink)); body.writeTo(gzipSink); gzipSink.close(); }}; }Copy the code

The lua script is used on the back-end Nginx to decompress and forward.

5.15 Breakpoint upload/Download

It uses the RANGE and content-range headers of HTTP, plus Java’s randomAccessFile API.

The main engineering problems are more difficult to deal with. Well written frameworks are rare. I didn’t do the resumable function here.

5.16 Post-download Processing

Copy some of the thunderbolt download software functions, provided in the form of API

Such as:

Verify MD5 / SHA1 after downloading
Open automatically after download: You need to handle the File URI permission of Android7
Notify Mediastore to scan after downloading
Hide files or not: For downloading private files, you know. Use. Nomedia empty file hide, prevent gentleman does not prevent villains.
Notification bar displays download progress/Dialog box displays download progress

5.17 Callback format

callback
livedata
Returns the observables

5.18 Interface Aggregation

Scenario 1 Multiple images are uploaded asynchronously

public static io.reactivex.Observable<ResponseBean<S3Info>> uploadImgs(String type, final List<String> filePaths){ final List<S3Info> infos = new ArrayList<>(); io.reactivex.Observable<ResponseBean<S3Info>> observable = HttpUtil.requestAsJsonArray(getUploadTokenPath,S3Info.class) .get() .addParam("type", type) .addParam("contentType", IMAGE_JPEG) .addParam("cnt",filePaths.size()) .asObservable() .flatMap(new Function<ResponseBean<List<S3Info>>, ObservableSource<ResponseBean<S3Info>>>() { @Override public ObservableSource<ResponseBean<S3Info>> apply(ResponseBean<List<S3Info>> bean) throws Exception { infos.addAll(bean.bean); List<io.reactivex.ObservableSource<ResponseBean<S3Info>>> observables = new ArrayList<>(); for(int i = 0; i< bean.bean.size(); i++){ S3Info info = bean.bean.get(i); String filePath = filePaths.get(i); io.reactivex.Observable<ResponseBean<S3Info>> observable = HttpUtil.request(info.getUrl(),S3Info.class) .uploadBinary(filePath) .put() .setExtraFromOut(info) .responseAsString() .treatEmptyDataAsSuccess() .asObservable(); observables.add(observable); } return io.reactivex.Observable.merge(observables); }}); return observable; }Copy the code

Scenario 2: Multiple interfaces make asynchronous requests with one callback

Background micro services are too fine, and do not want to do aggregation, can only be done by the client.

On the client side, the general aggregation interface request is implemented based on Rxjava.

Each interface can be configured to accept failures

code