A, problem,

This year, nullPointerExceptions of Gson and Kotlin Data Class were found online for several times. I did not study them carefully before, but only dealt with the parameters with problems in order to fix the problem. Recently I happened to have some time, and found that this kind of problem appeared in the company’s project many times, so I did a research on the cause of this problem, and sort out the treatment plan.

1. Parameters do not have default values

Let’s look at an example where there is no default constructor value

data class Bean(val id:Int.val name:String)

val json = "{\n \"id\": 100\n}"
val beanGson = GsonBuilder().create().fromJson(json,Bean::class.java)
Log.i("gson_bean_0"."id:${beanGson.id}; name:${beanGson.name}")
Copy the code

The Bean needs id and name2 parameters, but the Json only has id parameter. Guess what the result will be?

I/gson_bean_0: id:100; name:null
Copy the code

That’s a little weird. Isn’t name set to a non-null String? Why print null? Let’s first look at the results of Bean decompilation

public final class Bean {
   private final int id;
   @NotNull
   private final String name;

   public final int getId(a) {
      return this.id;
   }

   @NotNull
   public final String getName(a) {
      return this.name;
   }

   public Bean(int id, @NotNull String name) {
      Intrinsics.checkNotNullParameter(name, "name");
      super(a);this.id = id;
      this.name = name;
   }
   // omit tostring, hashcode, equals, etc
}
Copy the code

In the Bean’s constructor, kotlin’s null-safely check was performed for name. Why didn’t the NPE fire when Gson parsed? What magic did it use to get around it? Here first hang a hook 1 ️, wait until the following reasons to explore the chapter to explain together.

2. All parameters have default values

Now we add the default parameters to both ID and name, leaving the rest unchanged

data class Bean(val id:Int=1.val name:String="idtk")

val json = "{\n \"id\": 100\n}"
val beanGson = GsonBuilder().create().fromJson(json,Bean::class.java)
Log.i("gson_bean_1"."id:${beanGson.id}; name:${beanGson.name}")

// log
I/gson_bean_1: id:100; name:idtkCopy the code

Json does not return a specific name value, but you can see that the parameter defaults are in effect. Now let’s look at the difference between the decomcompiled Bean class and the one without the default values

public final class Bean {
   private final int id;
   @NotNull
   private final String name;

   public final int getId(a) {
      return this.id;
   }

   @NotNull
   public final String getName(a) {
      return this.name;
   }

   public Bean(int id, @NotNull String name) {
      Intrinsics.checkNotNullParameter(name, "name");
      super(a);this.id = id;
      this.name = name;
   }

   // $FF: synthetic method
   public Bean(int var1, String var2, int var3, DefaultConstructorMarker var4) {
      if ((var3 & 1) != 0) {
         var1 = 1;
      }

      if ((var3 & 2) != 0) {
         var2 = "idtk";
      }

      this(var1, var2);
   }

   public Bean(a) {
      this(0, (String)null.3, (DefaultConstructorMarker)null);
   }

	  // omit tostring, hashcode, equals, etc
}
Copy the code

In contrast to the decompilation with no default value, the Bean class has a no-argument constructor, which is something to watch and love, and you’ll see why when you see the source code later. Now for another experiment, what if I explicitly specified name as null in Json?

data class Bean(val id:Int=1.val name:String="idtk")

val json = "{\n \"id\": 100,\n \"name\": null\n}"
val beanGson = GsonBuilder().create().fromJson(json,Bean::class.java)
Log.i("gson_bean_2"."id:${beanGson.id}; name:${beanGson.name}")
Copy the code

You can guess what will happen:

1. Throw NullPointerException

2. Print name as IDTK

3. Print null for name

I/gson_bean_2: id:100; name:null
Copy the code

Kotlin’s null-safely check did not work. If kotlin’s Safely check did not work, the user’s name was Safely saved. Here we attach the second hook, 2 discount ️, which will be explained in the following reason exploration section.

3. The parameter part has default values

Now let’s set some of the parameters to their default values and see what happens

data class Bean(val id:Int=1.val name:String)

val json = "{\n \"id\": 100\n}"
val beanGson = GsonBuilder().create().fromJson(json,Bean::class.java)
Log.i("gson_bean_3"."id:${beanGson.id}; name:${beanGson.name}")

val json = "{\n \"id\": 100,\n \"name\": null\n}"
val beanGson = GsonBuilder().create().fromJson(json,Bean::class.java)
Log.i("gson_bean_4"."id:${beanGson.id}; name:${beanGson.name}")

// log

I/gson_bean_3: id:100; name:null
I/gson_bean_4: id:100; name:null
Copy the code

This is similar to the first case where there is no default value, so I won’t go into too much detail here, but let’s go into Gson’s source code and explore the reasons for these parsing results.

Second, to explore the reasons

Gson fromJson approach, general is according to the data type, select the corresponding TypeAdapter to parse the data, the above example for Bean object, will eventually go to ReflectiveTypeAdapterFactory. The create method, Return TypeAdapter, which invokes the constructorConstructor. Get (type) method, here basically to look at it

public <T> ObjectConstructor<T> get(TypeToken<T> typeToken) {
    final Type type = typeToken.getType();
    final Class<? super T> rawType = typeToken.getRawType();

    // omit some code...

    ObjectConstructor<T> defaultConstructor = newDefaultConstructor(rawType);
    if(defaultConstructor ! =null) {
      return defaultConstructor;
    }

    ObjectConstructor<T> defaultImplementation = newDefaultImplementationConstructor(type, rawType);
    if(defaultImplementation ! =null) {
      return defaultImplementation;
    }

    // finally try unsafe
    return newUnsafeAllocator(type, rawType);
  }
Copy the code

There are three ways to create objects

1. The newDefaultConstructor method attempts to create an object using the no-parameter constructor function. Returns the object if the object is successfully created, or null if it is not.

Object[] args = null;
return (T) constructor.newInstance(args);
Copy the code

Type 2, newDefaultImplementationConstructor method, through reflection collections framework to create the object, the above example is not the kind of this situation.

Unsafe, the newUnsafeAllocator method, builds objects from the sun.misc.Unsafe allocateInstance method. The Unsafe class gives Java the ability to manipulate data directly in memory. For more on broadening, check out Meituan’s article, “Java Magic Classes: Parsing Parsing parsing Unsafe.”

Class<? > unsafeClass = Class.forName("sun.misc.Unsafe");
Field f = unsafeClass.getDeclaredField("theUnsafe");
f.setAccessible(true);
final Object unsafe = f.get(null);
final Method allocateInstance = unsafeClass.getMethod("allocateInstance", Class.class);
return new UnsafeAllocator() {
  @Override
  @SuppressWarnings("unchecked")
  public <T> T newInstance(Class<T> c) throws Exception {
    return(T) allocateInstance.invoke(unsafe, c); }};Copy the code

I think readers already know the answer to the two hooks in chapter 1 after looking at the above three methods of constructing objects.

The first hook 1 ️

Unsafe, the data object doesn’t have a safe-handed constructor, so the Unsafe safe-handed scheme is the only option for handling the memory-fetched object. The Unsafe constructor bypits the null-safely check, so it doesn’t throw an NPE. The parameters in Chapter 1 with no default values and some parameters with default values can be classified in this case.

The second hook 2 ️

In that case, the data object will be Safely used to build the object without any arguments, and Gson set the property using reflection, so kotlin’s null-safely check won’t be triggered, so no NPE will be thrown.

Third, solutions

To solve the above Json parsing problem, I have compiled the following two solutions for you to choose from.

1, choosemoshi

Moshi is an open source library provided by Square that provides support for the Kotlin Data Class. Simple use as follows:

val moshi = Moshi.Builder()
	// Add the adapter that Kotlin parses
	.add(KotlinJsonAdapterFactory())
  .build()

val adapter = moshi.adapter(Bean::class.java)
val bean = adapter.fromJson(json)?:return
Log.i("gson_bean_5"."${bean.id}:${bean.name}")
Copy the code

Moshi validates Json parameters that explicitly return NULL, and throws a JsonDataException if the parameter cannot be null. JsonDataException is also thrown if a field is missing from Json and the default value is not set for that field.

Moshi’s GitHub address

2. CustomizeGsontheTypeAdapterFactory

The Gson framework can intervene in the Json data parsing process by adding a TypeAdapterFactory. We can write a custom TypeAdapterFactory to complete our support for Kotlin Data Class. We need to achieve the following objectives:

• For parameters whose type cannot be NULL and whose default value is set, if this field is missing in the Json or is explicitly null, the default value is used instead
• For parameters whose type cannot be NULL and whose default value is not set, an exception is thrown if the field is missing from the Json or if the field is explicitly null
• Parameters whose type can be NULL, regardless of whether the default value is set, can be resolved if the field is missing in the returned Json, or if the field is explicitly null

For the above requirements, the default value of the object needs to be obtained first, and then the processing steps are as follows according to whether 1 ️ parameter is NULL, 2 discount ️ parameter and 3 parameter free constructor of data:

1. Check whether it is a Kotlin object. If not, skip it. If yes, continue

   private val KOTLIN_METADATA = Metadata::class.java
   
   // If the class is not Kotlin, do not use a custom type adapter
   if(! rawType.isAnnotationPresent(KOTLIN_METADATA))return null
   Copy the code

2. The object is created using the no-argument constructor, and the default value of the object is cached

   val rawTypeKotlin = rawType.kotlin
   // No argument constructor
   val constructor= rawTypeKotlin.primaryConstructor ? :return null
   constructor.isAccessible = true
   // Params is mapped to value
   val paramsValueByName = hashMapOf<String, Any>()
   // Check whether the null argument is constructed
   val hasNoArgs = rawTypeKotlin.constructors.singleOrNull {
       it.parameters.all(KParameter::isOptional)
   }
   if(hasNoArgs ! =null) {
       // Construct an instance with no arguments
       val noArgsConstructor = (rawTypeKotlin as KClass<*>).createInstance()
       rawType.declaredFields.forEach {
           it.isAccessible = true
           val value = it.get(noArgsConstructor) ? :return@forEach
           paramsValueByName[it.name] = value
       }
   }
   Copy the code

3. During deserialization, determine whether the parameters of the data class can be null, whether the values read in the serialization are NULL, and whether the parameters have cached values

   1. Arguments can be null, and if serialization reads null, continue
   2. Arguments can be null, serialization reads a value that is not null, continue
   3. The parameter cannot be null. If the value read by serialization is not NULL, continue
   4. The parameter cannot be null. The value read by serialization is NULL. If the parameter has a cached default value, set the parameter to the default value
   5. The parameter cannot be null, the value read by serialization is NULL, the parameter does not have a cached default value, and an exception is thrown

   val value: T? = delegate.read(input)
   if(value ! =null) {
       /** * If the argument cannot be null, null is converted to the default value. If there is no default value, an exception */ is thrown
       rawTypeKotlin.memberProperties.forEachIndexed { index, it ->
           if(! it.returnType.isMarkedNullable && it.get(value) == null) {
               val field = rawType.declaredFields[index]
               field.isAccessible = true
               if(paramsValueByName[it.name] ! =null) {
                   field.set(value, paramsValueByName[it.name])
               } else {
                   throw JsonParseException(
                       "Value of non-nullable member " +
                               "[${it.name}] cannot be null")}}}}return value
   Copy the code

Disadvantages of this scheme

• Now, can this solution be perfect? I don’t know if anyone noticed the second step in the steps, but to implement this scenario, you must have a no-argument constructor, assumingkotlin data classFailing to do so? Now let’s combine thatGsonThe third of three constructorsUnsafePlan to think together. At this point, because there is no no-argument constructor, the data object will passUnsafeThe object is created and the data type is given the default value by the VM. In this case, when serializing, the result read by the basic type will not be null, but will be the default value assigned by the VIRTUAL machine, thus avoiding the check.

Four,

When parsing Kotlin’s data class, Gson uses the Unsafe scheme to create an object if the data does not provide the default no-argument constructor. In that case, the Unsafe scheme will skip Kotlin’s null-safely check, and the data values in the object are the initial values assigned by the VM. Instead of the default we defined, we first need to supply the object with a no-argument constructor. But even with no parameters, if the returned Json, explicitly specify a parameter is null, we still powerless, can access at this time I provide KotlinJsonTypeAdapterFactory above, it will check whether the parameter can be null, if not null, The default value is used to replace NULL.

This scenario is not perfect and requires you to provide a Kotlin Data class with no argument constructors to ensure that nullPointerExceptions are not raised.

KotlinJsonTypeAdapterFactory warehouse address

If you have any questions during the reading, please feel free to contact me.

Blog:www.zhichaoma.com

GitHub: github.com/Idtk

Email address:[email protected]