This article was first published in the Vivo Internet technology WeChat public number
Links:
https://mp.weixin.qq.com/s/CL4MsQEsrWS8n7lhXCOQ_g
By Li Bingyan
This paper mainly describes an ORM encapsulation practice of native SQLite database, which brings some enlightenment and reference significance to the business scenarios using native database operation (such as: it is an SDK itself), as well as a deeper understanding of database operation, APT, generics and other concepts by following the implementation ideas of the framework.
Implementation idea: SQL patchwork is carried out by obtaining request interface parameters through dynamic proxy, and RawType and ActualType of interface return value (generic) type are adapted to call mode and execution result, so as to encapsulate the actual SQL operation in its internal to simplify the purpose of database operation.
The background,
There is no doubt that there are many popular ORM frameworks for Android database, such as Room, Greendao, DBFlow and so on, which provide simple and easy to use API. Especially, Google open source Room is the most mainstream framework at present.
Why wrap your own database framework when you already have so many? Absolutely not for normal APP development, and one of these frameworks is perfect for your everyday needs; But if you are an SDK developer and your business is a database-dependent scenario, if you cannot rely on a third-party SDK (maintainability, troubleshooting, stability, size), then you will have to write native SQLite operations yourself. This can be a tedious and error-prone process (database upgrade/downgrade/open/close, multithreading cases, patching SQL statements, ContentValues interpolation data, cursor traversal/close, Entity transformations, etc.).
In order to avoid the above tedious and error-prone problems in the SDK development scenario, the following series of thinking and transformation are followed.
II. Expected Purpose
- Can simplify the native add, delete, change and check lengthy operations, do not go to write error-prone intermediate logical steps
- Automatically generate database build, upgrade/downgrade logic
- Easy-to-use call interface (support for synchronous/asynchronous, thread switching)
- Stable and reliable, no performance problems
III. Plan research
Observation of our daily business code can be found: a database query and a network request in the process is very similar, are through the construction of the request, request, intermediate steps, obtain results, processing results and other steps. Therefore, it is felt that the database operation can be abstracted and encapsulated in the form of network request. The detailed comparison is shown in the following table:
By comparing the above similarities and considering the entry point in conjunction with the existing ORM framework, the first thing that comes to mind is to use annotations:
The mainstream Room uses compile-time annotations (which are more conducive to performance), but in the process of specific coding and implementation of the Processor, I found that the input and output processing of add, delete, change and check operations was a little too cumbersome (refer to Room implementation), which was not suitable for the scene that is itself an SDK, and finally passed.
Runtime annotation processing are relatively simple (interface and parameter easier adaptation, processing can also write directly we are familiar with android native code), and the front has a famous library network request Retrofit using runtime annotations implementation example of network request, so you can as well try to realize the database to add and delete operations, It is also the final realization plan of this transformation.
I’m sure most Android client developers have used Retrofit. Here’s how it works: Use dynamic proxy to get the Method object corresponding to the interface as the entry, and use various parameters of the Method object (annotation modification) to construct a Request object and throw it to OkHttp to make the actual Request, and the return value is adapted to the Request result (bean object) and the call Method through Convert and Adapter, such as: Call<List<Bean>>, Observable<List<Bean>>, etc.
In this way, it encapsulates the internal details of the network request, greatly simplifying the network request process. Depending on their similarity, database operations (add, delete, change and search) can also be further encapsulated using this mechanism.
For such error-prone steps as database table building, upgrade, and demotion, it is best not to let the user write this part of logic manually. The solution uses compile-time annotations (Entitiy class and field attribute and version number correspond to each other through annotations), and automatically generates the implementation class of SQLiteOpenHelper during compilation.
By taking the above two parts into account, all the operation of pain points did not require the caller to pay attention to (just focus on passing the parameter and returning the result). Therefore, it was independent into a database module, named “Sponsor” ([KOMP].
4. Sponsor call example
Definition of Entity:
//Queryable: A Queryable object, which is Queryable over a Cursor, is converted to an Entitiy by bool convert(Cursor Cursor). There's the method contentValues convert(), ContentValues public class FooEntity implements Queryable, Insertable {/** * private int id */ private int id; /** * entitiy id */ private String fooId; /** * Entity */ private String data; // other attributes //getter()/setter()}2, interface definition, declaration add delete change check interface:
/** * Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities @Entities entities); @Query("SELECT * FROM "+ fooEntity. Table +" WHERE "+ fooEntity. Create_Time +" > ") + Parameter1.NAME + " AND " + FooEntity.CREATE_TIME + " < " + Parameter2.NAME + " ORDER BY " + FooEntity.CREATE_TIME + " ASC LIMIT " + Parameter3.NAME) Call<List<FooEntity>> queryEntitiesByRange(@Parameter1 long start, @Parameter2 long end, @Parameter3 int limit); /** * Delete * @return Delete number of records */ @delete (tableName = fooEntity. Table, fooEntity. Table, fooEntity. whereClause = FooEntity.ID + " >= " + Parameter1.NAME + " AND " + FooEntity.ID + " <= " + Parameter2.NAME) Call<Integer> deleteByIdRange(@Parameter1 int startId, @Parameter2 int endId);Create FooService create FooService
AllowMainThreadQueries () // Is running on the main thread operation? The default does not allow / /. AddCallAdapterFactory (RxJava2CallAdapterFactory. Create ()) / / rxjava //.addCallAdapterFactory(Java8CallAdapterFactory.create()) //java8 / /. AddCallAdapterFactory (LiveDataCallAdapterFactory. The create ()) / / livedata logger (new SponsorLogger ()) / / log output. The build (); Fooservice mFooservice = Sponsor.create (Fooservice.class); Fooservice mFooservice = Sponsor.create (Fooservice.class);Insert Entitiy data:
// ArrayList< fooEntity entities = new ArrayList<>(); // Entities = entities // Entities = entities // Entities = entities mFooService.insertEntities(entities).execute(); . / / asynchronous mFooService insertEntities (entities). The enqueue (new Callback < Integer > () {@ Override public void onResponse(Call<Integer> call, Integer rowId) { //success } @Override public void onFailure(Call<Integer> call, Throwable t) { //failed } });SQL > select a list of Entitiy objects from a list of Entitiy objects
List<FooEntity> entities; / / entities for the query result set entities. = mFooService queryEntitiesByRange (1, 200, 100). The execute ();Return the total number of records deleted:
/ / cout to delete a few int count = mFooService. DeleteByIdRange (0, 100). The execute ();Note:
- All of the above operations support synchronous/asynchronous invocation depending on the specific scenario.
- Add, delete, change operation Call
return value parameter (generic parameter) can also be directly specified as Throwable. If an internal exception can be returned through it, it is null on success
Five, the core implementation point
The basic principle is still based on the implementation of Retrofit framework, which gets various parameters of Method object through dynamic proxy to perform SQL piecing, and ADAPTS the execution results through Converter and Adapter. The overall framework is composed of the following modules:
- Sponsor: Principal implementation
- Sponsor_annotaiton: Annotations definitions, including runtime and compile-time annotations
- Sponsor_compiler: Processor implementation of database table creation, upgrade/downgrade logic, etc
- Sponsor_JAVA8, Sponsor_Livedata, Sponsor_RXJAVA2: ADAPTS several major invocation modes
1, dynamic proxy entry
public <T> T create(final Class<T> daoClass, final Class<? extends DatabaseHelper> helperClass) {
final Object obj = Proxy.newProxyInstance(daoClass.getClassLoader(), new Class<?>[]{daoClass},
new InvocationHandler() {
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
if (method.getDeclaringClass() == Object.class) {
return method.invoke(this, args);
}
DaoMethod<Object, Object> daoMethod =
(DaoMethod<Object, Object>) loadDaoMethod(method);
final DatabaseHelper helper = loadDatabaseHelper(daoClass, helperClass);
Call<Object> call = new RealCall<>(helper, mDispatcher, mAllowMainThreadQueries,
mLogger, daoMethod, args);
return daoMethod.adapt(call);
}
});
return (T) obj;
}2. Interface Adaptation
Since the dynamic proxy will return the Method object of the interface and the parameter list args[], all the elements identified above can be obtained through these two parameters. The specific Method is as follows:
Get method annotations: Method. GetAnnotations () to obtain parameter list: has been coming for parameter annotation and type: method. The getParameterAnnotations () method. The getGenericParameterTypes () call way: Method. GetGenericReturnType (), and then Call the getRawType () / / Call to obtain the result Type: Method. GetGenericReturnType (), and then call the getActualTypeArguments () / / the List < FooEntitiy >3. Return result adaptation
private Converter<Response, ? > createQueryConverter(Type responseType, Class<? > rawType) { Converter<Response, ? > converter = null; If (Queryable. Class. IsAssignableFrom (rawType)) {/ / return a single entity object / / other processing logic converter = new QueryableConverter ((class <? extends Queryable>) responseType); } else if (rawType == list.class) {// Converter = new ListQueryableConverter((class <? extends Queryable>) argumentsTypes[0]); } else if (rawType == Integer.class) {SELECT COUNT(*) FROM table to Converter = new IntegerConverter(); } else if (rawType == Long.class) { converter = new LongConverter(); } return converter; }ListQueryableConverter implementation, which traverses Cursor to build a list of results returned:
static final class ListQueryableConverter implements Converter<Response, List<? extends Queryable>> {
@Override
public List<? extends Queryable> convert(Response value) throws IOException {
List<Queryable> entities = null;
Cursor cursor = value.getCursor();
if (cursor != null && cursor.moveToFirst()) {
entities = new ArrayList<>(cursor.getCount());
try {
do {
try {
//反射创建entitiy对象
Queryable queryable = convertClass.newInstance();
final boolean convert = queryable.convert(cursor);
if (convert) {
entities.add(queryable);
}
} catch (Exception e) {
e.printStackTrace();
}
} while (cursor.moveToNext());
} catch (Exception e) {
e.printStackTrace();
}
}
/*
* 避免返回null
*/
if (entities == null) {
entities = Collections.emptyList();
}
return entities;
}
}4. Execute the operation of adding, deleting, changing and checking
Final class RealCall<T BB0 implements Call<T> {@Override public T Execute () {/** * Final class RealCall<T BB0 implements Call<T> {@Override public T Execute () {/** * Final class RealCall<T BB0 implements Call<T> {@Override public T Execute () {/** * perform(); T value = null; try { value = mDaoMethod.toResponse(response); } catch (Exception e) { e.printStackTrace(); } finally {// turn off if (response! = null) { Cursor cursor = response.getCursor(); if (cursor ! = null) { try { cursor.close(); } catch (Exception e) { e.printStackTrace(); }}} // Database closes if (MDatabaseHelper! = null) { try { mDatabaseHelper.close(); } catch (Exception e) { e.printStackTrace(); } } } return value; } /** * private Response Perform () {switch (MDaomeThod.getAction ()) {case Action.query: {/ /.. Cursor cursor = query(String sql); } case Actions.DELETE: { //... int count = delete(simple, sql, tableName, whereClause); } case Actions.INSERT: { //... } case Actions.UPDATE: { //... } } return null; } /** * private Cursor query(String SQL) {//... SQLiteDatabase db = mDatabaseHelper.getWritableDatabase(); final Cursor cursor = db.rawQuery(sql, null); / /... return cursor; } /** * private int delete(Boolean Simple, String SQL, String TableName, String TableName, String TableName) String whereClause) { SQLiteDatabase db = mDatabaseHelper.getWritableDatabase(); int result = 0; try { db.beginTransaction(); / /... result = db.delete(tableName, whereClause, null); db.setTransactionSuccessful(); } finally { try { db.endTransaction(); } catch (Throwable t) { t.printStackTrace(); } } return result; }}VI. Comparison of performance tests
- Test phone: Vivo X23
- Android version: Android 9
- Processor: Snapdragon 670, 2.0GHz, 8 cores
- Test Method: Test 5 sets of data for each comparison item, 5 rounds of tests for each group, and then take the average (rounded)
Description:
- The fourth test in the table (found all 10W data) has a large difference (79ms difference). The reason is that the Entity object of the native interface is directly new, while Sponsor can only reflect and create through the NewInstance () interface of the Entity, which leads to the performance gap. However, when calculated on average, With 1000 objects created per newInstance(), only 1ms more is created, so the impact is still small. (Tried to optimize the Clone method, but the effect was still not obvious)
- The Sponsor approach has slightly lower performance than the native approach because it requires the performance cost of dynamically piecing together SQL statements, but the cost is minimal.
VII. Application practices in the project (SDK)
The database used in this project is a multi-database and multi-table architecture. The database operations (add, delete, modify, search, build tables, upgrade/downgrade, etc.) are all written by the code logic using the native interface of SQLiteOpenHelper. As a result, the related operations need to write a lot of template codes to get the final result, and the logic is quite lengthy. Therefore, in the refactored version, we used sponsor to replace these native calls, in order to simplify these cumbersome and error-prone operations. At present, the operation is good, no obvious serious problems have been found.
Extension of knowledge – type erasure of generics
About type erasements, I feel that many people have some misunderstandings, especially the client development usually involves less, I don’t feel very understanding:
Common sense dictates that Java generics are type-erased at runtime, so there is no specific type information (Object or some upper bound type) after compilation.
The question then becomes, how does Retrofit get the method generic parameter types (both parameter types and return types) at run time, given that the types are erased? For example, you can internally convert JSON to the corresponding bean object based on the return type of a function.
It was also difficult to understand at first, so I basically understood it after searching materials, communicating with technical groups, and writing demo verification, and summed it up in one sentence: Type erasure simply erases the generic parameter (for compatibility with versions 1.5 and below). The original bytecode still retains the generic Type information of the class structure (class, method, field), which is stored in the Signature section. The generic Type information can be retrieved at runtime using the subclass interface of Type.
1. Retrofit request interfaces are generally defined as follows:
You can see that both the return type and the parameter type of this function take generic parameters.
2. Decompile the APK and open it with the JD-GUI tool as follows:
Many people will look at this and think that the generic type information has indeed been completely erased. However, this tool only shows simple class structure information (including only classes, functions, and fields), we can take a closer look at the corresponding class ByteCode to make sure, directly open apk with AS, expand classes.dex to find the corresponding class, right-click ->”Show ByteCode” to see:
You can see that in the Signature field all the parameter information for this method is stored, including the generic type information.
Any declaration of a class, interface, constructor method, or field that contains a generic type will generate the Signature property for which the generic Signature information is recorded, but the generic information for local variables within the function will not be recorded.
3. Let’s take a look at the inheritance relationship of the Type interface and the interface functions provided:
Class: The most common type, an object of a Class Class represents a Class or interface in a virtual machine.
ParameterizedType: ParameterizedType: ParameterizedType: ParameterizedType: List
, Map
with generics.
- Type getRawType() — Returns the primitive Type of a parameterized Type, such as List
whose primitive Type is List.
- Type[] getActualTypeArguments() — Gets the Type variable of a parameterized Type, or a list of actual types, such as Map
is Integer and String.
TypeVariable: Represents type variables, such as T in List
.
GenericArrayType: Indicates an array type with elements that are parameterizedType or TypeVariable, such as List
or T[].
- Type getGenericComponentType() A method that returns the component types of an array.
WildcardType: Represents the WildcardType, such as? And extends the Number? Super Integer. Common methods are:
- Type[] getUpperBounds() — Returns the upper bound of a Type variable.
- Type[] getLowerbounds () — Returns the lower boundary of a Type variable.
9. Reference materials
- https://github.com/square/retrofit
- https://cs.android.com/androidx/platform/frameworks/support/+/android-room-release:room/compiler/src/main/kotlin/android x/room/processor/
- https://techblog.bozho.net/on-java-generics-and-erasure/
- https://blog.csdn.net/u011983531/article/details/80295479
More content please pay attention to Vivo Internet technology WeChat public number
Note: Please contact WeChat ID: Labs2020 to reprint the article.