preface

Core Data is an efficient database framework for iOS (although Core Data is not a database, it still uses Sqlite3 to store Data underneath). It can manipulate Data as objects, and developers don’t need to care about how Data is stored on disk. It’s going to take a managed object that’s in the NS-managed Object Context an instance of nS-managed Object class or an instance of an NS-managed object subclass, managed object Model managed object model, Keep the managed objects to the persistent store coordinator NSPersistentStoreCoordinator hold one or more persistent store NSPersistentStore. The queries that use Core Data are optimized by Apple, so they are efficient queries.

When you do simple things like setting the default value of an entity, setting the cascading deletion operation, setting the validation rules for Data, using the Data request template, Core Data does all of these things itself, without having to migrate the Data itself. What operations will require data migration? Any changes that will cause the managed object model of NS-Managed ObjectModel should be migrated to prevent users from backing off after upgrading the application. Ns-managed ObjectModel changes the managed object model by adding a table, adding an entity in a table, adding an attribute of an entity, and migrating an attribute of an entity to an attribute of another entity ………… You should now know which operations require data migration.

Tip:

So before WE get into that, I’m going to talk about three debugging things that you might need to do in Core Data.

1. Generally, there are three types of files in app sandbox: SQLite, SQlite-shm, and SQlite-wal. The last two are generated by a new “Database Journaling Mode” enabled by the system by default after iOS7. Sqlite-shm is a Shared Memory file that contains an index of sqlite-wal. The SHM file is automatically generated by the system, so if you delete it, it will be generated again next time. Sqlite-wal is a write-ahead Log file that contains uncommitted database transactions. If there are sqlite-wal files in the database, there are still uncommitted transactions in the database. If you open the SQLite file again, it is likely that the last database operation has not been performed.

Therefore, when debugging, we need to observe the changes of the database in real time, we can disable the logging mode first, just need to create a persistent store when the parameter can be saved. The specific code is as follows

    NSDictionary *options =
    @{
          NSSQLitePragmasOption: @{@"journal_mode": @"DELETE"}
     };
    
    NSError *error = nil;
    _store = [_coordinator addPersistentStoreWithType:NSSQLiteStoreType
                                        configuration:nil
                                                  URL:[self storeURL]
                                              options:options error:&error];
Copy the code

2. There are many ways to open the database on Mac. I recommend 3 ways. Of course, there are also friends who do not use Firefox, for example, I am a heavy user of Chrome, so I recommend two free small apps, one is SQliteBrowser, the other is SQLite Manager, these two are relatively lightweight, relatively easy to use.

3. If you want to see how Core Data optimizes your queries underneath, here’s a way to do it.

First go to Product ->Scheme ->Edit Scheme




And then switch to the Arguments in the paging in the Arguments Passed On Launch add “- the com. Apple. CoreData. SQLDebug 3”, rerun the app, the following will display the Core Data optimized Sql statements.




Ok, the debugging information should be perfectly displayed, and you can happily enter the body!

Core Data comes with lightweight Data migration

This migration is not to be underestimated, you must add it when creating a new table, otherwise you will get the following error,

**Failed to add store. Error: Error Domain=NSCocoaErrorDomain Code=134100 "(null)" UserInfo={metadata={** ** NSPersistenceFrameworkVersion = 641; ** ** NSStoreModelVersionHashes = {** ** Item = <64288772 72e62096 a8a4914f 83db23c9 13718f81 4417e297 293d0267 79b04acb>; ** ** Measurement = <35717f0e 32cae0d4 57325758 58ed0d11 c16563f2 567dac35 de63d5d8 47849cf7>; * * * *}; ** ** NSStoreModelVersionHashesVersion = 3; ** ** NSStoreModelVersionIdentifiers = (** ** ""** ** ); ** ** NSStoreType = SQLite; ** ** NSStoreUUID = "9A16746E-0C61-421B-B936-412F0C904FDF"; ** ** "_NSAutoVacuumLevel" = 2; ** **}, reason=The model used to open the store is incompatible with the one used to create the store}**Copy the code

Reason =The model used to open The store is incompatible with The one used to create The store, just because I created a new table, But I didn’t turn on the lightweight migration Option. There will be some people here will ask, I new table never appear this error ah? That’s because the third-party framework you’re using has already written the Option. (Sideline: Who writes Core Data from zero these days? I’m sure they all use third-party frameworks.) If you are writing Core Data from 0, you will get an error. The solution is to add code and take advantage of the lightweight migration of Core Data to prevent the flashback problem of not finding the storage area


NSDictionary *options =
    @{
      NSSQLitePragmasOption: @{@"journal_mode": @"DELETE"},
      NSMigratePersistentStoresAutomaticallyOption :@YES,
      NSInferMappingModelAutomaticallyOption:@YES
    };
    
    NSError *error = nil;
    _store = [_coordinator addPersistentStoreWithType:NSSQLiteStoreType
                                        configuration:nil
                                                  URL:[self storeURL]
                                              options:options error:&error];
Copy the code

Here is the meaning of the two new parameters: NSMigratePersistentStoresAutomaticallyOption = YES, then the Core Data, try to put before the emergence of the low version is not compatible with the persistent storage area to migrate to the new model, the example, the Core Data can identify is the new table, The new table will be created in the storage area, there will be no error above.

NSInferMappingModelAutomaticallyOption = YES, this parameter is the meaning of the Core Data will be according to your think is the most reasonable way to try MappingModel, from the source a property of the entity model is mapped to the target model entities of a property.

Then we look at MagicRecord source code is how to write, so we can perform some operations will not appear I said above the flash back problem


+ (NSDictionary *) MR_autoMigrationOptions;
{
    // Adding the journalling mode recommended by apple
    NSMutableDictionary *sqliteOptions = [NSMutableDictionary dictionary];
    [sqliteOptions setObject:@"WAL" forKey:@"journal_mode"];
    
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
                             [NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption,
                             sqliteOptions, NSSQLitePragmasOption,
                             nil];
    return options;
}
Copy the code

The above section is MagicRecord source code for you to add Core Data lightweight Data migration protection, so you do not write the 2 parameters, the same will not report an error. If you unlog those two parameters, or set the parameter value to NO, run it again, and create a new table, you will get the error I mentioned above. You can practice, after all, the real knowledge comes from practice.

Open as long as the two parameters above, the Core Data will perform their own lightweight migration, of course, in the real property transfer, unreliable in the way, before I think it is sure to infer, the result or direct flash back an error after update later, may be because the table structure is too complex, more than the simple inference ability range, so I suggest, When moving complex entity attributes to another attribute, don’t put too much faith in this approach and it’s best to map yourself. Of course, when you create a new table, these two parameters must be added!!

Core Data Manually creates a Mapping file for migration

This is a little more subtle than the previous approach. The Mapping file specifies which attributes of the entity are migrated to which attributes of the entity. This is more reliable than the first approach, which leaves it up to Core Data to infer. First of all, what can go wrong if you don’t include this Mapping file in a complex migration


**Failed to add store. Error: Error Domain=NSCocoaErrorDomain Code=134140 "(null)" UserInfo={destinationModel=(<NSManagedObjectModel: 0x7f82d4935280>) isEditable 0, entities {**
**    Amount = "(<NSEntityDescription: 0x7f82d4931960>) name Amount, managedObjectClassName NSManagedObject, renamingIdentifier Amount, isAbstract 0, superentity name (null), properties {\n    qwe = \"(<NSAttributeDescription: 0x7f82d4930f40>), name qwe, isOptional 1, isTransient 0, entity Amount, renamingIdentifier qwe, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 700 , attributeValueClassName NSString, defaultValue (null)\";\n}, subentities {\n}, userInfo {\n}, versionHashModifier (null), uniquenessConstraints (\n)";**
**    Item = "(<NSEntityDescription: 0x7f82d4931a10>) name Item, managedObjectClassName Item, renamingIdentifier Item, isAbstract 0, superentity name (null), properties {\n    collected = \"(<NSAttributeDescription: 0x7f82d4930fd0>), name collected, isOptional 1, isTransient 0, entity Item, renamingIdentifier collected, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 800 , attributeValueClassName NSNumber, defaultValue 0\";\n    listed = \"(<NSAttributeDescription: 0x7f82d4931060>), name listed, isOptional 1, isTransient 0, entity Item, renamingIdentifier listed, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 800 , attributeValueClassName NSNumber, defaultValue 1\";\n    name = \"(<NSAttributeDescription: 0x7f82d49310f0>), name name, isOptional 1, isTransient 0, entity Item, renamingIdentifier name, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 700 , attributeValueClassName NSString, defaultValue New Item\";\n    photoData = \"(<NSAttributeDescription: 0x7f82d4931180>), name photoData, isOptional 1, isTransient 0, entity Item, renamingIdentifier photoData, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 1000 , attributeValueClassName NSData, defaultValue (null)\";\n    quantity = \"(<NSAttributeDescription: 0x7f82d4931210>), name quantity, isOptional 1, isTransient 0, entity Item, renamingIdentifier quantity, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 600 , attributeValueClassName NSNumber, defaultValue 1\";\n}, subentities {\n}, userInfo {\n}, versionHashModifier (null), uniquenessConstraints (\n)";**
**}, fetch request templates {**
**    Test = "<NSFetchRequest: 0x7f82d49316c0> (entity: Item; predicate: (name CONTAINS \"e\"); sortDescriptors: ((null)); type: NSManagedObjectResultType; )";**
**}, sourceModel=(<NSManagedObjectModel: 0x7f82d488e930>) isEditable 1, entities {**
**    Amount = "(<NSEntityDescription: 0x7f82d488f880>) name Amount, managedObjectClassName NSManagedObject, renamingIdentifier Amount, isAbstract 0, superentity name (null), properties {\n    abc = \"(<NSAttributeDescription: 0x7f82d488f9d0>), name abc, isOptional 1, isTransient 0, entity Amount, renamingIdentifier abc, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 700 , attributeValueClassName NSString, defaultValue (null)\";\n}, subentities {\n}, userInfo {\n}, versionHashModifier (null), uniquenessConstraints (\n)";**
**    Item = "(<NSEntityDescription: 0x7f82d488fbe0>) name Item, managedObjectClassName NSManagedObject, renamingIdentifier Item, isAbstract 0, superentity name (null), properties {\n    collected = \"(<NSAttributeDescription: 0x7f82d48901c0>), name collected, isOptional 1, isTransient 0, entity Item, renamingIdentifier collected, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 800 , attributeValueClassName NSNumber, defaultValue 0\";\n    listed = \"(<NSAttributeDescription: 0x7f82d488fd20>), name listed, isOptional 1, isTransient 0, entity Item, renamingIdentifier listed, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 800 , attributeValueClassName NSNumber, defaultValue 1\";\n    name = \"(<NSAttributeDescription: 0x7f82d488fdb0>), name name, isOptional 1, isTransient 0, entity Item, renamingIdentifier name, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 700 , attributeValueClassName NSString, defaultValue New Item\";\n    photoData = \"(<NSAttributeDescription: 0x7f82d488fad0>), name photoData, isOptional 1, isTransient 0, entity Item, renamingIdentifier photoData, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 1000 , attributeValueClassName NSData, defaultValue (null)\";\n    quantity = \"(<NSAttributeDescription: 0x7f82d488fc90>), name quantity, isOptional 1, isTransient 0, entity Item, renamingIdentifier quantity, validation predicates (\\n), warnings (\\n), versionHashModifier (null)\\n userInfo {\\n}, attributeType 600 , attributeValueClassName NSNumber, defaultValue 1\";\n}, subentities {\n}, userInfo {\n}, versionHashModifier (null), uniquenessConstraints (\n)";**
**}, fetch request templates {**
**    Test = "<NSFetchRequest: 0x7f82d488fa60> (entity: Item; predicate: (name CONTAINS \"e\"); sortDescriptors: ((null)); type: NSManagedObjectResultType; )";**
**}, reason=Can't find mapping model for migration}**
Copy the code

Can’t find the mapping Model for migration, so create a mapping Model file.

In the same folder as your XCDatamodeld, “New File “->” Core Data”->”Mapping Model”


Select the source database that you want to map




Then select the target database





Then name the Mapping Model file





As a note, it is best to have a name that at first glance identifies which database version is being upgraded. I wrote ModelV4ToV5, so it is a V4 to V5 upgrade.

The importance of the Mapping file is explained here. First, you should add a Mapping file between databases of an earlier version. In this way, when upgrading a database of an earlier version, errors of each version will not occur and users will not blink back after the upgrade.


Such as above, there is a Mapping between each database file, V0ToV1, V1ToV2, V2ToV3, V3ToV4, V4ToV5, must each Mapping.

For example, if the user is on the old version of V3, due to the update rules of appstore, each update is directly updated to the latest version, then the user will be directly updated to V5. If either of the intermediate V3ToV4 and V4ToV5 is missing, then the V3 user will not be able to upgrade to V5 and will be backlocked. This is why it is important to add a Mapping file between each version. In this way, users of any earlier version can upgrade to the latest version through the Mapping file at any time without blinking back.

Now let’s talk about what the Mapping file open is.


Opening the Mapping file corresponds to the Source entity attribute, and migrating to the Target entity attribute Mapping. The top is the attribute, and the bottom is the relationship Mapping. $source is the source entity represented

This makes it very clear how Core Data lightweight migration differs from manually creating a Mapping for migration so far. 1.Core Data lightweight migration is suitable for adding new tables, adding new entities, adding new entity attributes, and other simple migration methods that the system can deduce by itself. 2. Manually creating the Mapping is suitable for more complex data migration

For example, let’s say I started with a very abstract table called Object, which stores properties of things, and let’s say I have name, width, height. Suddenly, I had a new requirement. I needed to add several new fields in the Object table, such as colour, weight, etc. Since these are simple new fields and do not involve data transfer, lightweight migration can be used at this time.

But suddenly a program has a new requirement, need to add two tables, one is Human table, one is Animal table, need to define the original abstract Object table more concrete. In this case, we need to extract all the people in Object and put them in the new “Human” table, and also extract all animals and put them in the new “Animal” table. Since both new tables have name attributes, the system may not be able to infer which names should be in the Human table and which should be in the Animal table if a lightweight migration is performed at this point. Also, there are some properties in the Human table that are not in the Animal table. This is the time to manually add a Mapping Model file that specifies which attributes are attributes of the source entity and which attributes should be mapped to the target entity. This kind of more refined migration can only be done manually by adding the Mapping Model, after all, iOS does not know your needs and ideas.

Data migration through code

This migration through code is mainly in the process of data migration, if you want to do something else, such as you want to clean up garbage data, real-time data migration progress, etc., that needs to be implemented here.

First, we need to check whether the storage exists, and then compare the model metadata in the storage area, check whether it is compatible, if not compatible, then we need to carry out data migration.

- (BOOL)isMigrationNecessaryForStore:(NSURL*)storeUrl { NSLog(@"Running %@ '%@'", self.class, NSStringFromSelector(_cmd)); if (! [[NSFileManager defaultManager] fileExistsAtPath:[self storeURL].path]) { NSLog(@"SKIPPED MIGRATION: Source database missing."); return NO; } NSError *error = nil; NSDictionary *sourceMetadata = [NSPersistentStoreCoordinator metadataForPersistentStoreOfType:NSSQLiteStoreType URL:storeUrl error:&error]; NSManagedObjectModel *destinationModel = _coordinator.managedObjectModel; if ([destinationModel isConfiguration:nil compatibleWithStoreMetadata:sourceMetadata]) { NSLog(@"SKIPPED MIGRATION: Source is already compatible"); return NO; } return YES; }Copy the code

When the above function returns YES, we need to merge, so the next function is the one below

- (BOOL)migrateStore:(NSURL*)sourceStore { NSLog(@"Running %@ '%@'", self.class, NSStringFromSelector(_cmd)); BOOL success = NO; NSError *error = nil; // STEP 1 - Collect Source entities, Destination target entities and the Mapping Model file NSDictionary * sourceMetadata = [NSPersistentStoreCoordinator metadataForPersistentStoreOfType:NSSQLiteStoreType URL:sourceStore error:&error]; NSManagedObjectModel *sourceModel = [NSManagedObjectModel mergedModelFromBundles:nil forStoreMetadata:sourceMetadata]; NSManagedObjectModel *destinModel = _model; NSMappingModel *mappingModel = [NSMappingModel mappingModelFromBundles:nil forSourceModel:sourceModel destinationModel:destinModel]; // STEP 2 - Start migration merge if the Mapping Model is not empty or if (mappingModel) {NSError *error = nil; NSMigrationManager *migrationManager = [[NSMigrationManager alloc] initWithSourceModel:sourceModel destinationModel:destinModel]; [migrationManager addObserver:self forKeyPath:@"migrationProgress" options:NSKeyValueObservingOptionNew context:NULL]; NSURL *destinStore = [[self applicationStoresDirectory] URLByAppendingPathComponent:@"Temp.sqlite"]; success = [migrationManager migrateStoreFromURL:sourceStore type:NSSQLiteStoreType options:nil withMappingModel:mappingModel toDestinationURL:destinStore destinationType:NSSQLiteStoreType destinationOptions:nil error:&error]; If (success) {/ / STEP 3 - with a new migrated store to replace the old store the if ([self replaceStore: sourceStore withStore: destinStore]) { NSLog(@"SUCCESSFULLY MIGRATED %@ to the Current Model", sourceStore.path); [migrationManager removeObserver:self forKeyPath:@"migrationProgress"]; } } else { NSLog(@"FAILED MIGRATION: %@",error); } } else { NSLog(@"FAILED MIGRATION: Mapping Model is null"); } return YES; // Migration is complete}Copy the code

In the above function, if the migration progress has changed, the user is informed of the progress through the observer, observeValueForKeyPath, which can monitor the progress and, if it has not been completed, prohibit the user from performing certain actions

- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context { if ([keyPath isEqualToString:@"migrationProgress"]) { dispatch_async(dispatch_get_main_queue(), ^{ float progress = [[change objectForKey:NSKeyValueChangeNewKey] floatValue]; int percentage = progress * 100; NSString *string = [NSString stringWithFormat:@"Migration Progress: %i%%", percentage]; NSLog(@"%@",string); }); }}Copy the code

Of course, this merge data migration operation must be performed with a multi-threaded asynchronous, so as not to cause user interface lag, and then add the following method, we come to asynchronous execution

- (void)performBackgroundManagedMigrationForStore:(NSURL*)storeURL { NSLog(@"Running %@ '%@'", self.class, NSStringFromSelector(_cmd)); dispatch_async( dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{ BOOL done = [self migrateStore:storeURL]; if(done) { dispatch_async(dispatch_get_main_queue(), ^{ NSError *error = nil; _store = [_coordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:[self storeURL] options:nil error:&error]; if (! _store) { NSLog(@"Failed to add a migrated store. Error: %@", error); abort(); } else { NSLog(@"Successfully added a migrated store: %@", _store); }}); }}); }Copy the code

At this point, the data migration is complete, but there is still a question is, when should we perform the migration operation, after the update? The appDelegate comes in? The best way is to perform data migration before adding the current storage area to the coordinator.

- (void)loadStore { NSLog(@"Running %@ '%@'", self.class, NSStringFromSelector(_cmd)); if (_store) {return; } // Do not load again, because already loaded BOOL useMigrationManager = NO; if (useMigrationManager && [self isMigrationNecessaryForStore:[self storeURL]]) { [self performBackgroundManagedMigrationForStore:[self storeURL]]; } else { NSDictionary *options = @{ NSMigratePersistentStoresAutomaticallyOption:@YES ,NSInferMappingModelAutomaticallyOption:@YES ,NSSQLitePragmasOption: @{@"journal_mode": @"DELETE"} }; NSError *error = nil; _store = [_coordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:[self storeURL] options:options error:&error]; if (! _store) { NSLog(@"Failed to add store. Error: %@", error); abort(); } else { NSLog(@"Successfully added store: %@", _store); }}}Copy the code

This completes the data migration and shows the progress of the migration. You can also customize some operations during the migration, such as cleaning up garbage data, deleting unused tables, and so on.

The end of the

Ok, here, I have shared with you several ways of Core Data migration. If there is any wrong place in the article, welcome to mention it, and we can communicate and make progress together!