What is the DataStore

It’s described on the website

A data storage solution. Use Kotlin coroutines and flows to store data asynchronously, consistently, and transactionally

“If you currently use SharedPreferences to store data, consider migrating to DataStore.”

From the description on the official website, you can roughly tell that DataStore does much the same thing as SharedPreferences. Both store small amounts of data.

Why DataStore

On the official website, it is explicitly recommended that we migrate to DataStore rather than continue to use SharedPreferences.

Let’s start with an official summary

function	SharedPreferences	Preferences DataStore	Proto DataStore
asynchronous	✅(only used to read changed values) via the listener	✅ through Flow	✅ through Flow
synchronous	✅	❌	❌
Safely call the UI thread	❌	✅	✅
Missignal	❌	✅	✅
Not affected by runtime exceptions	❌	✅	✅
Transactional apis with strong consistency assurance	❌	✅	✅
Handling data migration	❌	✅	✅
Type safety	❌	❌	✅

The table is from the Android website

By the above summary. You can see that. SharedPreferences has many security risks compared with DataStore. In Jetpack, Google gave me the impression that it was trying to keep all the errors in-house, from Kotlin’s syntactic sugar. ViewBinding is a particular security risk for developers. It’s also a good thing.

How does the Preferences DataStore work

On the official website, DataStore provides two different implementations

Preferences DataStoreUse keys to store and access data. This implementation does not require a predefined schema and does not provide type safety.
Proto DataStoreStore data as instances of custom data types. This implementation requires you to define schemas using protocol buffers, but it provides type safety.

This section starts with a look at how the Preferences DataStore is used.

Add the dependent

I summarize its use for the Preferences DataStore. The first step is definitely to add dependencies

// Preferences DataStore (SharedPreferences like APIs) dependencies { implementation "Androidx. Datastore: datastore - preferences: 1.0.0 - alpha05"} / / no use Android environment. Dependencies {implementation "Androidx. Datastore: datastore - preferences - core: 1.0.0 - alpha05"}Copy the code

Android developers really only need to add the above line. The latest version is still alpha. Use it sparingly in projects.

Create the DataStore

The same from the official website. Create a DataStore using createDataStore, an extension of Context, where name is mandatory.

val dataStore: DataStore<Preferences> = context.createDataStore(
  name = "allens_form_ds"
)
Copy the code

As with SharePreferences, name corresponds to the name of a file, as shown below.

Add attributes

Since Datastore objects are SharePreferences. How do you add attributes to SharePreferences

 getSharedPreferences("allens_form_sp", Context.MODE_PRIVATE)
      .edit()
      .putString("name"."Waiting for you in the rainy season.")
      .putInt("age".22)
      .commit()
Copy the code

Take a look at the DataStore.

GlobalScope.launch {
	// Get the DataStore object getSharedPreferences
      val dataStore: DataStore<Preferences> = createDataStore(
          name = "allens_form_ds"
      )
      / / create a key
      val userName = preferencesKey<String>("name")
      val userAge = preferencesKey<Int>("age")
      dataStore
          .edit { value ->
              value[userName] = "Waiting for you in the rainy season."
              value[userAge] = 22}}Copy the code

And SharePreferences distinction

It is almost the same as SharePreferences. Points to be aware of

Key needs to be usedpreferencesKeyorpreferencesSetKeyTo create
Edit is a suspend function that needs to be used in a coroutine

The same as SharePreferences

The same point as SharePreferences. Both support only basic data types, which can also be peeked at in the preferencesKey method

public inline fun <reified T : Any> preferencesKey(name: String): Preferences.Key<T> {
    return when (T::class) {
        Int::class -> {
            Preferences.Key<T>(name)
        }
        String::class -> {
            Preferences.Key<T>(name)
        }
        Boolean::class -> {
            Preferences.Key<T>(name)
        }
        Float::class -> {
            Preferences.Key<T>(name)
        }
        Long::class -> {
            Preferences.Key<T>(name)
        }
        Double::class -> {
            Preferences.Key<T>(name)
        }
        Set::class -> {
            throw IllegalArgumentException("Use `preferencesSetKey` to create keys for Sets.")}else- > {throw IllegalArgumentException("Type not supported: ${T::class.java}")}}}Copy the code

Retrieve attributes

DataStore uses a Flow stream to obtain attributes. If you are not familiar with the Flow, you can spend 10 minutes on the official website to get a general impression of the example in the section. Click on the asynchronous Flow

The datastore. data method returns a Flow. You can process the Flow yourself. We can obtain the attributes we want by key, or map the Flow and other operations to transform the saved attributes.

Gets a property

dataStore.data
	// You can use the Flow collect attribute to obtain the Flow and judge in the Flow
    .collect {
    	// Get the corresponding value by passing in the key
        println("user name:${it[userName]}")}Copy the code

Get multiple attributes

dataStore.data
    .collect {
        println("user name:${it[userName]}")
        println("user age:${it[userAge]}")}Copy the code

You can also do this using the Flow onEach operator

dataStore.data
    .onEach {
        println("user name:${it[userName]}")
        println("user age:${it[userAge]}")
    }
    .collect()
Copy the code

Exception handling

Exception control is something that belongs to Flow control Flow. For this article, a bit of procrastination, the author here is simply taken over.

dataStore.data
    .onEach {
        println("user name:${it[userName]}")
        println("user age:${it[userAge]}")
        // Simulate an exception
        throw NullPointerException()
    }
    // Handle exceptions
    .catch { println("error $it")}// Success and failure go here. Similar to finly
    .onCompletion { println("complete") }
    .collect()
Copy the code

In short, you can do a lot of things with Flow, you can convert to map, etc. It is still recommended to take a look at the Flow DataStore in hand first

Proto DataStore using

Before using Proto DataStore, there is still a question of why the DataStore needs it now that it has Preferences.

Why Proto DataStore

When learning the Preferences DataStore in the previous section, we found that Proto DataStore only supports some basic data types and cannot be customized. Proto DataStore was born to solve this problem.

Configuration depends on

Proto DataStore is more complex to use than the Preferences DataStore. The following author also sorted out its configuration process as follows. Easy for readers to use and configure

addProtobufThe plug-in

Configure the plugin in your app’s build.gradle. If you depend on other modules, the child modules need to do the following.

plugins {
    id "com.google.protobuf" version "0.8.12"
}
Copy the code

Add the dependent

dependencies {
    //Android only needs to add this one
    implementation  "Androidx. Datastore: datastore: 1.0.0 - alpha05"
    / / Protobuf dependency
    implementation  "Com. Google. Protobuf: protobuf - javalite: 3.14.0"
}
Copy the code

Configuration Protobuf

Add the following nodes in build.gradle. Note that it is the same level node as Android

Protobuf / / configuration
protobuf {
    protoc {
        artifact = "Com. Google. Protobuf: protoc: 3.10.0"
    }

    // Generates the java Protobuf-lite code for the Protobufs in this project. See
    // https://github.com/google/protobuf-gradle-plugin#customizing-protobuf-compilation
    // for more information.
    generateProtoTasks {
        all().each { task ->
            task.builtins {
                java {
                    option 'lite'
                }
            }
        }
    }
}
Copy the code

Create a Protobuf object

Create the user_fs. Proto file under app/ SRC /main/proto

// Language version
syntax = "proto3";

/ / package name
option java_package = "com.allens.okdatastore";
option java_multiple_files = true;

// Each new type is denoted by message
message UserPreferences {
  bool show_completed = 1;
  int32 age = 2;
  string name = 3;
  float price = 4;
}
Copy the code

Be sure to rebuild once you’re done. The compiler will help me create usable Java objects

For those of you who may find this user_prefs.proto very unreadable, we recommend a plugin for Android Studio

The code doesn’t look that ugly. At least it has a color

Proto language

The user_fs. Proto file defined above uses the Proto language, which is not that deep for Android development. For more information, see the Proto language guide

Protocol Buffers (ProtocolBuffer/ Protobuf) is a data description language developed by Google. Similar to XML, it serializes structured data and can be used for data storage and communication protocols. Protocol Buffers have many advantages over XML in serializing structured data:

More simple
The data description file is only 1/10 to 1/3 of the original size
Parsing is 20 to 100 times faster
It reduces ambiguity
Generate data access classes that are easier to use programmatically

serialization

To tell the DataStore how to read and write the data types defined in the original file, we need to implement a Serializer. Create a new kind of UserPreferencesSerializer

Note that the UserPreferences class may not be found without rebuild

object UserPreferencesSerializer : Serializer<UserPreferences> {
    override val defaultValue: UserPreferences = UserPreferences.getDefaultInstance()
    override fun readFrom(input: InputStream): UserPreferences {
        try {
            return UserPreferences.parseFrom(input)
        } catch (exception: InvalidProtocolBufferException) {
            throw CorruptionException("Cannot read proto.", exception)
        }
    }

    override fun writeTo(t: UserPreferences, output: OutputStream) = t.writeTo(output)
}
Copy the code

Write data

val dataStore = createDataStore(
    fileName = "user_prefs.pb". serializer = UserPreferencesSerializer ) dataStore.updateData { preferences -> preferences.toBuilder() .setShowCompleted(false)
        .setAge(22)
        .setName("River ocean")
        .setPrice(10.5 f)
        .build()
}
Copy the code

Read the data

val dataStore = createDataStore(
    fileName = "user_prefs.pb",
    serializer = UserPreferencesSerializer
)

dataStore.data
    .collect {
        println("age:${it.age}")
        println("name:${it.name}")
        println("price:${it.price}")
        println("completed:${it.showCompleted}")}Copy the code

SharedPreferences migration

It’s also very simple to use

createDataStore(
  // The name of the migration to DataStore
  name = "user",
  migrations = listOf(
      SharedPreferencesMigration(
          context = this@MainActivity.// The original SharedPreferences name
          sharedPreferencesName = "user")))Copy the code

It is important to note that the migration does not take place immediately after execution, but requires a read or write operation to take effect. The original SharedPreferences will be deleted after the successful migration

Some thoughts on DataStore

advantages

First of all. I strongly agree with DataStore’s design philosophy that you should not simply set a String parameter as a key like SharedPreferences. Instead, it forces a preferencesKey to be associated with the set type. Google has been trying to help developers reduce the number of crashes caused by error types.

Secondly, Flow is introduced as an event Flow. Helps developers handle events better. It’s like writing RxJava.

Finally, DataStore methods are forced to be used in coroutines. It also prevents blocking threads from making safe calls to the UI thread

Based on the appeal of 3 points, the author in the process of learning that its design ideas, as well as the SharedPreferences of 0 cost migration. It is possible to replace SharedPreferences

disadvantages

I can’t praise her too much because she is alpha so far. Compared to SharedPreferences, it can be said that the API is relatively difficult to use, and basically wins. How does that compare to MMKV?

Readers familiar with MMKV may know this. (If you are not familiar with it, please refer to the MMKV Chinese document on the official website.) MMKV is a key-value component based on MMAP memory mapping. Protobuf is used to realize the underlying serialization/deserialization, which has high performance and strong stability

DataStore, as you can see above, is still essentially stored as a file. And when you use custom properties. I’m going to write Proto. It is also more troublesome. Using MMKV requires only one line of code. This is a complete failure. What is even more hateful is that MMKV also supports SharedPreferences migration.

Compared to actual development projects, I think MMVK does have an advantage over DataStore. Its API and its MMAP approach are better suited for small amounts of data storage.

Problem with DataStore actually found

The 2021-1-22 update

There’s been a misidentification here. This design is a very good design. It is the author’s own thoughts that remain at the level of SharedPreferences. It’s like the DataStore author is already at level 10. I’m still on the second floor. To judge the library. The following is the author’s reply.

Data storage. Data is a stream that should not be completed. It should be emitted every time there is a change in the data store.Copy the code

Flow itself listens for changes in the data. And the author thinks it is a mistake. A disgrace.

Here I write a small example

suspend fun main(a) {(1.3.).asFlow()
        .onCompletion { println("Complete") }
        .collect { println("$it")}}Copy the code

The above code execution, I believe we all know

One, two, three, doneCopy the code

The Flow Flow should then end. But DataStore in version 1.0.0-alpha05 actually found that Flow could not end

🌰

I start by creating a DataStore and adding a name attribute. I’m going to leave the code out here and query the name (I’m using runBlocking here). Don’t use this for real projects.)

viewBinding.linear.addView(createButton("Test") {
    runBlocking {
        val dataStore: DataStore<Preferences> = createDataStore(
            name = "test"
        )
        val key = preferencesKey<String>("name")
        dataStore.data
            .onCompletion { println("Complete") }
            .collect { println("${it[key]}")}}})Copy the code

According to the idea, the name should be printed and finished, which is the same as 123 above. The actual result is that the coroutine hangs and does not end after the name is printed. It’s not clear why. If you know, you can tell me.

OKDataStore

DataStore usage is described in the preceding sections. Here are some of my small packages for DataStroe. The author found. DataStore’s original design was to bind return types to key strengths. Prevent errors. The intention was good, but convenience was sacrificed. Even if MMKV is used, only a key of String type is needed to save attributes. In the author’s opinion, if only a simple data type is saved. Don’t need so can use, but a bit gild the lily.

Save the data

OKDataStore mimics SharedPreferences to add and save data as follows.

runBlocking {
    createOkDataStore("user")
        .edit()
        .putString("name"."River ocean")
        .putInt("age".21)
        .putBoolean("isBody".true)
        .putFloat("size".20f)
        .putLong("long".100L)
        .putStringSet("setData", setOf("a"."b"."c"."d"))
        .commit()

}
Copy the code

For custom types, I still use Proto DataStore. In a real project where you really want to have some custom properties, you can use JSON to store strings. So I’m not too excited about this one either

runBlocking {
    val dataStore = createOkDataStore(
        fileName = "user_prefs.pb". serializer = UserPreferencesSerializer ) dataStore.updateData { preferences -> preferences.toBuilder() .setShowCompleted(false)
            .setAge(22)
            .setName("River ocean")
            .setPrice(10.5 f)
            .build()
    }
}
Copy the code

To get the data

To obtain data, the author considers that in general, the data to be obtained is not so complex, but there are requirements that may need to set the default option, after all, SharedPreferences and MMKV can be set in this way. The benefits of this design are obvious, so the default option is included in OKDataStore. And you can get the specified type. Flow is still retained. The author believes that Flow is relatively easy to use in Kotlin.

runBlocking {
    createOkDataStore("user")
    	.getInt("age".10000)
        .collect {
            println("data:$it")}}Copy the code

If it is an error type, the catch operator can be used to handle it, for example

runBlocking {
    createOkDataStore("user")
    	// write on purpose so that there is a type error
    	.getInt("name".10000).catch {
            println(Error :${it. Message})
        }
        .collect {
            println("data:$it")}}Copy the code

You can also use the Flow feature directly to process all the results yourself, with the same effect as datastore.data. The reader is left to his own devices, but the problem shown in the previous section, Flow, does not end

runBlocking {
    createOkDataStore("user")
    	.flow()
        .collect {
            println("data:$it")}}Copy the code

Getting custom types is still DataStore style

runBlocking {
    val dataStore = createOkDataStore(
        fileName = "user_prefs.pb",
        serializer = UserPreferencesSerializer
    )
    dataStore.flow()
        .collect {
            println("age:${it.age}")
            println("name:${it.name}")
            println("price:${it.price}")
            println("completed:${it.showCompleted}")}}Copy the code

To resolve the problem of DataStore not being able to terminate Flow. The author has customized a Throwable to interrupt Flow. Of course, this option can also be turned off. By default, it is turned on. Simply set Cancel to false to close, leaving the reader to deal with the Flow problems. As to why. The author here is not very clear. BUG? Or is the design? Have the small partner that knows can inform!

fun Context.createOkDataStore(
    name: String,
    cancel: Boolean = true,
    migrations: List<DataMigration<Preferences>> = listOf()
): OKDataStore {
    return OKDataStoreImpl(name = name, context = this, cancel = cancel, migrations = migrations)
}
Copy the code

Design ideas

I found that DataStore design actually has only one event flow and then returns a Preferences. By retrieving the saved parameters in Preferences with the key, I stream this event through the flatMapConcat operator into multiple event streams. And then go to the corresponding flow.

private fun <T> getValueFormKey(key: String, default: T): Flow<T> {
    return dataStore.data
        .map { it.asMap() }
        .flatMapConcat { requestFlow(it, key, default) }
        .cancellable()
        .map { it.second as T }
}
Copy the code

After all, it is the flatMapConcat operator that is in preview state. Well, god knows what will happen next.

Finally put the project source code, you can have a look. OKDataStore

DataStore usage