This article belongs to “Jane Book — Liu Xiaozhuang” original, please note:
A Simple Book — Liu Xiaozhuang
NSURLSession
NSURLSession was introduced in iOS7. NSURLSession was introduced to replace NSURLConnection, which is simpler to use and doesn’t have to deal with runloop-related stuff.
In 2015, THE RFC 7540 standard released HTTP 2.0, which contains many new features and significant improvements in transmission speed. NSURLSession starts with iOS9.0 and supports HTTP 2.0.
NSURLSession consists of three parts:
- NSURLSession: Request a session object, either using a system-provided singleton or creating it yourself.
- NSURLSessionConfiguration:
sessionSession configuration is generally adopteddefault. - NSURLSessionTask: Responsible for executing a specific request
taskBy thesessionTo create.
NSURLSession can be created in three ways:
sharedSession
Copy the codeA singleton maintained by the system that can share connection and request information with other tasks using this session.
sessionWithConfiguration:
Copy the codeUpon initial NSURLSession pass in a NSURLSessionConfiguration, so we can custom request headers, cookies and other information.
sessionWithConfiguration:delegate:delegateQueue:
Copy the codeIf you want more control over the request process and the callback thread, you need the above method to initialize and pass in a delegate to set the callback object and the callback thread.
Making a network request via NSURLSession is also relatively simple.
- Create a NSURLSessionConfiguration configuration request.
- Create an NSURLSession object using Configuration.
- Initiate network requests through the Session object and obtain the Task object.
- Call the [Task Resume] method to initiate a network request.
NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
NSURLSession *session = [NSURLSession sessionWithConfiguration:config
delegate:self
delegateQueue:[NSOperationQueue mainQueue]];
NSURLSessionDataTask *task = [session dataTaskWithURL:[NSURL URLWithString:@"http://www.baidu.com"]];
[task resume];
Copy the codeNSURLSessionTask
Each request initiated by NSURLSession will be encapsulated as an NSURLSessionTask, but generally not directly NSURLSessionTask class, but based on the task type, is encapsulated as its corresponding subclass.
- NSURLSessionDataTask: process ordinary
Get,PostThe request. - NSURLSessionUploadTask: processes the upload request and can pass in the corresponding upload file or path.
- NSURLSessionDownloadTask: Handles the download address and provides the resumable function
cancelMethods.
The main methods are defined in the parent class NSURLSessionTask. Here are some of the key methods or properties.
CurrentRequest The current task being executed is generally the same as originalRequest unless redirected. OriginalRequest is used for redirection operations and records requests before redirection. TaskIdentifier Unique identifier of a task in the current session. Multiple sessions may have the same identifier. The priority can be set in the Priority Task, but this attribute does not represent the priority of the request, but rather identifies it. NSURLSession does not provide an API to change the priority of requests. State Indicates the status of the current task. You can use KVO to listen for status changes. – Resume Starts or continues the request. The created task is suspended by default. You need to manually invoke resume to start the request. – suspend suspends the current request. The main reason is that download requests are used more, and normal requests are restarted after being suspended. If the download request is suspended, calling Resume will continue the request as long as it does not exceed the timeout period set by NSURLRequest. – cancel Cancels the current request. Task will be marked as cancelled and sometime in the future call URLSession: task: didCompleteWithError: method.
NSURLSession provides a common way to create a task. After creating a task, you can override the proxy method to obtain the corresponding callback and parameters. This provides more control over the request process.
- (NSURLSessionDataTask *)dataTaskWithRequest:(NSURLRequest *)request;
- (NSURLSessionUploadTask *)uploadTaskWithRequest:(NSURLRequest *)request fromFile:(NSURL *)fileURL;
- (NSURLSessionDownloadTask *)downloadTaskWithRequest:(NSURLRequest *)request;
Copy the codeIn addition, NSURLSession also provides the block method to create tasks. The creation method is as simple as AFN, directly passing in the URL or NSURLRequest and receiving the returned data directly in the block. After a block is created, the state is suspend by default. You need to invoke Resume to start the task.
CompletionHandler and delegate are mutually exclusive, and the completionHandler has priority over the delegate. The block method is more result-oriented than the normal creation method, and you can get the result directly from the completionHandler, but you can’t control the request process.
- (NSURLSessionDataTask *)dataTaskWithURL:(NSURL *)url completionHandler:(void(^) (NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))completionHandler;
- (NSURLSessionUploadTask *)uploadTaskWithRequest:(NSURLRequest *)request fromData:(nullable NSData *)bodyData completionHandler:(void(^) (NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))completionHandler;
- (NSURLSessionDownloadTask *)downloadTaskWithURL:(NSURL *)url completionHandler:(void(^) (NSURL * _Nullable location, NSURLResponse * _Nullable response, NSError * _Nullable error))completionHandler;
Copy the codeYou can use the following two methods to obtain all tasks corresponding to the current session. The difference between the methods is that the callback parameters are different. GetTasksWithCompletionHandler, for example, in the AFN application is used to obtain the current session of the task, and will be AFURLSessionManagerTaskDelegate callback is set to nil, in order to prevent the collapse.
- (void)getTasksWithCompletionHandler:(void(^) (NSArray<NSURLSessionDataTask *> *dataTasks, NSArray<NSURLSessionUploadTask *> *uploadTasks, NSArray<NSURLSessionDownloadTask *> *downloadTasks))completionHandler;
- (void)getAllTasksWithCompletionHandler:(void(^) (NSArray<__kindof NSURLSessionTask *> *tasks))completionHandler);
Copy the codedelegateQueue
You can specify a thread when initializing an NSURLSession. If you don’t specify a thread, the completionHandler and delegate callback methods are executed in the child thread.
If NSURLSession is initialized with a delegateQueue, the callback is executed on the specified queue, and if mainQueue is specified, the callback is executed on the main thread, thus avoiding thread switching issues.
[NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:nil];
Copy the codedelegate
The proxy methods for NSURLSession will not be listed in detail here. The method names follow Apple’s principle of knowing what you mean by what you mean by what you mean, so it’s easy to use. Here is an introduction to the proxy inheritance structure of NSURLSession.
NSURLSession defines a series of proxies and follows the inheritance relationship described above. According to the inheritance relationship and proxy method declaration, if a task is performed, only one of the agents needs to be complied with.
Perform upload or ordinary Post request, for example, are followed NSURLSessionDataDelegate, download tasks follow NSURLSessionDownloadDelegate, the parent agency definition are all public methods.
Request redirection
The HTTP protocol defines a redirection status code, such as 301. The following proxy methods are used to process redirection tasks. You can either create a new request based on the response when the redirect occurs, or you can just use the system-generated request and pass it in the completionHandler callback, and if you want to terminate the redirect, pass nil to the completionHandler.
- (void)URLSession:(NSURLSession *)session
task:(NSURLSessionTask *)task
willPerformHTTPRedirection:(NSHTTPURLResponse *)response
newRequest:(NSURLRequest *)request
completionHandler:(void(^) (NSURLRequest *))completionHandler
{
NSURLRequest *redirectRequest = request;
if (self.taskWillPerformHTTPRedirection) {
redirectRequest = self.taskWillPerformHTTPRedirection(session, task, response, request);
}
if(completionHandler) { completionHandler(redirectRequest); }}Copy the codeNSURLSessionConfiguration
Create a way
NSURLSessionConfiguration responsible for NSURLSession initialization time, can be set by NSURLSessionConfiguration request cookies, key, caching, the first parameter, Separate some configuration parameters for network requests from NSURLSession.
NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
NSURLSession *session = [NSURLSession sessionWithConfiguration:config
delegate:self
delegateQueue:[NSOperationQueue mainQueue]];
Copy the codeNSURLSessionConfiguration provides three methods of initialization, below is some explanation of request method.
@property (class.readonly.strong) NSURLSessionConfiguration *defaultSessionConfiguration;
Copy the codeProvide defaultSessionConfiguration NSURLSessionConfiguration created, but this is not a single case method, but a kind of method, to create different objects. The configuration created in this way does not share cookies, cache, and keys. Each configuration needs to be set separately.
Many people on the Internet understand this is wrong and have not really used it in the project or have not paid attention to it. If there is any discrepancy with others, I will prevail.
@property (class.readonly.strong) NSURLSessionConfiguration *ephemeralSessionConfiguration;
Copy the codeCreate a temporary configuration. The main difference between an object created in this way and a normal object is URLCache, URLCredentialStorage, and HTTPCookieStorage. Likewise, Ephemeral is not a singleton method, but a class method.
URLCredentialStorage
Ephemeral <__NSCFMemoryURLCredentialStorage: 0x600001bc8320>
HTTPCookieStorage
Ephemeral <NSHTTPCookieStorage cookies count:0>
Copy the codeIf you print the Config created by the Ephemeral method, you will see that the variable type is significantly different from the other types and will be marked Ephemeral before the printed information. The Config created by Ephemeral does not generate persistent information and is well protected against data security for requests.
+ (NSURLSessionConfiguration *)backgroundSessionConfigurationWithIdentifier:(NSString *)identifier;
Copy the codeIdentifier mode is generally used to restore the previous task, mainly for download. If a download task is in progress and the program is called by kill, you can save the identifier before the program exits. The next task before entering the program through the identifier after recovery, system will NSURLSession and NSURLSessionConfiguration and associated download task before, and the task of before continuing.
timeout
timeoutIntervalForRequest
Copy the codeSet the timeout period between session requests. This timeout period is not the start and end of the request, but the interval between two packets. This value is reset when any request is returned, or times out if it is not returned within the timeout period. The unit is second. The default value is 60 seconds.
timeoutIntervalForResource
Copy the codeResource timeout duration, which is usually used for uploading or downloading tasks. It is timed after the uploading or downloading task starts. If the task is not finished, the resource file is deleted. The unit is second. The default time is seven days.
Resource sharing
If the same NSURLSessionConfiguration object, you will be sharing request header, cache, cookies and the Credential, created by the Configuration NSURLSession, will also have the corresponding request information.
@property (nullable.copy) NSDictionary *HTTPAdditionalHeaders;
Copy the codePublic request header, which is empty by default. Any NSURLSession configured by a Confuguration will carry the set request header.
@property (nullable.retain) NSHTTPCookieStorage *HTTPCookieStorage;
Copy the codeCookie manager for HTTP requests. If the NSURLSession is created using sharedSession or backgroundConfiguration, the Cookie data of sharedHTTPCookieStorage is used by default. If you don’t want to use cookies, you can just set them to nil, or you can manually set them to your own Cookie age.
@property (nullable.retain) NSURLCredentialStorage *URLCredentialStorage;
Copy the codeCertificate manager. If the NSURLSession is created using sharedSession or backgroundConfiguration, the certificate of sharedCredentialStorage is used by default. If you don’t want to use certificates, you can just set them to nil, or you can create your own certificate manager.
@property (nullable.retain) NSURLCache *URLCache;
Copy the codeRequest cache, nil if you don’t set it manually, NSURLCache is a class that I haven’t studied, I don’t know much about.
Cache handling
Can be set in the NSURLRequest cachePolicy request cache strategy, here no specific value is described in detail, the default value is NSURLRequestUseProtocolCachePolicy cache.
NSURLSessionConfiguration can set the processing of the cache object, we can manually set the cache object of custom, if it is not set, the default use system sharedURLCache singleton object cache. The NSURLRequest will use this NSURLCache for all requests made by the NSURLSession created by the Configuration.
@property (nullable.retain) NSURLCache *URLCache;
Copy the codeNSURLCache provides caches in Memory and Disk. When creating NSURLCache, you need to specify the size of Memory and Disk respectively and the location of files to be stored. Using NSURLCache, you do not need to worry about insufficient disk space or manually manage the memory space. If a memory warning occurs, the system will automatically clear the memory space. However, NSURLCache provides very limited functions, and projects rarely use it directly to process cached data, rather than using databases.
[[NSURLCache alloc] initWithMemoryCapacity:30 * 1024 * 1024
diskCapacity:30 * 1024 * 1024
directoryURL:[NSURL URLWithString:filePath]];
Copy the codeAnother advantage of using NSURLCache is that the server can set the expiration time of the resource. Upon requesting the server, the server will return cache-control indicating the expiration time of the file. NSURLCache automatically sets the expiration time based on NSURLResponse.
Maximum number of connections
Limit the maximum number of NSURLSession connections. The maximum number of nSURlSessions and server connections created using this method will not exceed the number set here. Apple set the default for us to be 4 on iOS and 6 on Mac.
@property NSInteger HTTPMaximumConnectionsPerHost;
Copy the codeConnection reuse
HTTP is based on the transport layer protocol TCP. When sending a network request through TCP, a three-way handshake is required. After establishing a network request, data is sent. Keep-alive is now supported in HTTP1.0. Keep-alive can keep established links. If the same domain name is used, subsequent requests will not be disconnected immediately after the connection is established, but will reuse existing connections. Keep-alive is enabled by default starting with HTTP1.1.
The following parameters are set in the request header. If the server supports keep-alive, it will also add the same fields in the response header to the client request.
Connection: Keep-Alive
Copy the codeIf you want to disconnect keep-alive, you can add the above and below fields to the request header, but this is generally not recommended.
Connection: Close
Copy the codeIf you use NSURLSession to make network requests, you need to use the same NSURLSession object. If you create a new session object, you cannot reuse the previous link. Keep-alive can hold a requested connection, and Apple allows a maximum of four connections to be held on iOS and six on Mac.
pipeline
In HTTP1.1, requests can also be piped based on keep-alive. The connection established with the same back-end service, the TCP layer, usually requires the previous request to return, the subsequent request to be sent. However, a pipeline can make subsequent requests without relying on the response from previous requests.
The pipeline relies on both the client and the server. After receiving the request from the client, the server will process and respond to the task in a first-in, first-out order. The pipeline still has a non-pipeline problem, which is that a problem with the previous request will block the current connection and affect subsequent requests.
Pipeline does not improve request speed for large files, only for normal requests. Can be set in the NSURLSessionConfiguration HTTPShouldUsePipelining to YES, open pipelines, this property is the default value is NO.
NSURLSessionTaskMetrics
In the daily development process, we often encounter the problem of slow page loading, which is largely due to the network. Therefore, it is a very important task to find and solve the cause of network time-consuming. Apple provides the NSURLSessionTaskMetrics class to check network metrics. NSURLSessionTaskMetrics is the delegate of the NSURLSessionTaskMetrics class. At the end of each task, the following methods are called back. And you can get a metrics object.
- (void)URLSession:(NSURLSession *)session
task:(NSURLSessionTask *)task
didFinishCollectingMetrics:(NSURLSessionTaskMetrics *)metrics;
Copy the codeNSURLSessionTaskMetrics can help us analyze the process of network requests to find out the cause of time consumption. In addition to the class, more detailed data on the NSURLSessionTaskTransactionMetrics class.
@property (copy.readonly) NSArray<NSURLSessionTaskTransactionMetrics *> *transactionMetrics;
Copy the codeEach element in the transactionMetrics array corresponds to a request for the current task. There is usually only one element in the array, and multiple elements may exist if redirects or other conditions occur.
@property (copy.readonly) NSDateInterval *taskInterval;
Copy the codeTaskInterval records the total time taken by a task from the beginning of a request to the end of the task. NSDateInterval includes startDate, endDate, and Duration.
@property (assign.readonly) NSUInteger redirectCount;
Copy the codeRedirectCount counts the number of redirects, which are typically performed during download requests to ensure that the download task is handled by the most appropriate back-end node.
NSURLSessionTaskTransactionMetrics
In NSURLSessionTaskTransactionMetrics attributes are used to do statistics, function is recording a value, and no logical sense. So here’s an explanation of some of the main attributes, basically covering most of the attributes, but not the rest.
This picture is I ripped off from the Internet, marked the NSURLSessionTaskTransactionMetrics attributes in the process of request is in what position.
// Request object
@property (copy.readonly) NSURLRequest *request;
// Response object, request failure may be nil
@property (nullable.copy.readonly) NSURLResponse *response;
// Request start time
@property (nullable.copy.readonly) NSDate *fetchStartDate;
// Start time of DNS resolution
@property (nullable.copy.readonly) NSDate *domainLookupStartDate;
// End time of DNS resolution, nil if resolution fails
@property (nullable.copy.readonly) NSDate *domainLookupEndDate;
// Start time to establish TCP connection
@property (nullable.copy.readonly) NSDate *connectStartDate;
// End time for establishing the TCP connection
@property (nullable.copy.readonly) NSDate *connectEndDate;
// Start TLS handshake time
@property (nullable.copy.readonly) NSDate *secureConnectionStartDate;
// End TLS handshake time
@property (nullable.copy.readonly) NSDate *secureConnectionEndDate;
// Start transmitting request data
@property (nullable.copy.readonly) NSDate *requestStartDate;
// End the transmission request data time
@property (nullable.copy.readonly) NSDate *requestEndDate;
// Time to receive server response data
@property (nullable.copy.readonly) NSDate *responseStartDate;
// The server responds to the data transfer completion time
@property (nullable.copy.readonly) NSDate *responseEndDate;
// Network protocols, such as HTTP /1.1
@property (nullable.copy.readonly) NSString *networkProtocolName;
// Request whether to use proxy
@property (assign.readonly.getter=isProxyConnection) BOOL proxyConnection;
// Whether to reuse existing connections
@property (assign.readonly.getter=isReusedConnection) BOOL reusedConnection;
// Resource identifier that indicates whether the request was loaded from Cache, Push, or Network
@property (assign.readonly) NSURLSessionTaskMetricsResourceFetchType resourceFetchType;
/ / local IP
@property (nullable.copy.readonly) NSString *localAddress;
// Local port number
@property (nullable.copy.readonly) NSNumber *localPort;
/ / the remote IP
@property (nullable.copy.readonly) NSString *remoteAddress;
// Remote port number
@property (nullable.copy.readonly) NSNumber *remotePort;
// TLS protocol version, if HTTP is 0x0000
@property (nullable.copy.readonly) NSNumber *negotiatedTLSProtocolVersion;
// Whether to use cellular data
@property (readonly.getter=isCellular) BOOL cellular;
Copy the codeHere is my HTTP download request, statistics obtained data. The device is Xcode simulator and the network environment is WiFi.
(Request) <NSURLRequest: 0x600000c80380> { URL: http://vfx.mtime.cn/Video/2017/03/31/mp4/170331093811717750.mp4 }
(Response) <NSHTTPURLResponse: 0x600000ed9420> { URL: http://vfx.mtime.cn/Video/2017/03/31/mp4/170331093811717750.mp4 } { Status Code: 200, Headers {
"Accept-Ranges" = (
bytes
);
Age = (
1063663
);
"Ali-Swift-Global-Savetime" = (
1575358696
);
Connection = (
"keep-alive"
);
"Content-Length" = (
20472584
);
"Content-Md5" = (
"YM+JxIH9oLH6l1+jHN9pmQ=="
);
"Content-Type" = (
"video/mp4"
);
Date = (
"Tue, 03 Dec 2019 07:38:16 GMT"
);
EagleId = (
dbee142415764223598843838e
);
Etag = (
"\"60CF89C481FDA0B1FA975FA31CDF6999\""
);
"Last-Modified" = (
"Fri, 31 Mar 2017 01:41:36 GMT"
);
Server = (
Tengine
);
"Timing-Allow-Origin" = (
"*"
);
Via = (
"Cache39. L2et2 [0200-0 H], cache6. L2et2 [3, 0], cache16. Cn548 [0200-0 H], cache16. Cn548 [1, 0]"
);
"X-Cache" = (
"HIT TCP_MEM_HIT dirn:-2:-2"
);
"X-M-Log" = (
"QNM:xs451; QNM3:71"
);
"X-M-Reqid" = (
"m0AAAP__UChjzNwV"
);
"X-Oss-Hash-Crc64ecma" = (
12355898484621380721
);
"X-Oss-Object-Type" = (
Normal
);
"X-Oss-Request-Id" = (
5DE20106F3150D38305CE159
);
"X-Oss-Server-Time" = (
130
);
"X-Oss-Storage-Class" = (
Standard
);
"X-Qnm-Cache" = (
Hit
);
"X-Swift-CacheTime" = (
2592000
);
"X-Swift-SaveTime" = (
"Sun, 15 Dec 2019 15:05:37 GMT"
);
} }
(Fetch Start) 2019- 12- 15 15:05:59 +0000
(Domain Lookup Start) 2019- 12- 15 15:05:59 +0000
(Domain Lookup End) 2019- 12- 15 15:05:59 +0000
(Connect Start) 2019- 12- 15 15:05:59 +0000
(Secure Connection Start) (null)
(Secure Connection End) (null)
(Connect End) 2019- 12- 15 15:05:59 +0000
(Request Start) 2019- 12- 15 15:05:59 +0000
(Request End) 2019- 12- 15 15:05:59 +0000
(Response Start) 2019- 12- 15 15:05:59 +0000
(Response End) 2019- 12- 15 15:06:04 +0000
(Protocol Name) http/1.1
(Proxy Connection) NO
(Reused Connection) NO
(Fetch Type) Network Load
(Request Header Bytes) 235
(Request Body Transfer Bytes) 0
(Request Body Bytes) 0
(Response Header Bytes) 866
(Response Body Transfer Bytes) 20472584
(Response Body Bytes) 20472584
(Local Address) 192.1681.105.
(Local Port) 63379
(Remote Address) 219.23820.101.
(Remote Port) 80
(TLS Protocol Version) 0x0000
(TLS Cipher Suite) 0x0000
(Cellular) NO
(Expensive) NO
(Constrained) NO
(Multipath) NO
Copy the codeFAQ
Why is the delegate of NSURLSession a strong reference?
After initializing the NSURLSession object and setting up the proxy, the proxy object will be strongly referenced. According to apple’s official comments, the strong hold does not have always existed, but the call URLSession: didBecomeInvalidWithError: methods, the delegate will be released.
By calling the NSURLSession invalidateAndCancel or finishTasksAndInvalidate method, the strong reference can be disconnected and execute didBecomeInvalidWithError: proxy approach, And then the session is invalid and can’t be used. That is, strong references can be broken only if the session is invalid.
Sometimes, in order to ensure connection reuse and other problems, it is not easy to invalid session, so it is better not to use NSURLSession directly, but to encapsulate it once and twice. This is one of the reasons for using AFN3.0.
NSURLSession uploads and downloads
File upload
Upload form
Sometimes the client needs to upload large files to the server, and the large files cannot be loaded into the memory at one go and passed to the server. When large files are uploaded, they are fragmented and uploaded one by one. It is important to note that shard upload and breakpoint continuation are not the same concept and upload does not support breakpoint continuation.
When uploading fragments, local files need to be read. NSFileHandle is used to read files. NSFileHandle provides an offset function. We can seek the current read position of handle to the last read position and set the read length of this time. The read file is the byte of the file we specify.
- (NSData *)readNextBuffer {
if (self.maxSegment <= self.currentIndex) {
return nil;
}
if(!self.fileHandler){
NSString *filePath = [self uploadFile];
NSFileHandle *fileHandle = [NSFileHandle fileHandleForReadingAtPath:filePath];
self.fileHandler = fileHandle;
}
[self.fileHandler seekToFileOffset:(self.currentIndex) * self.segmentSize];
NSData *data = [self.fileHandler readDataOfLength:self.segmentSize];
return data;
}
Copy the codeThe main method for uploading files is form uploading, namely multipart/from-data. AFNetworking also supports form uploading. Form uploads need to follow the following format, boundary is a hexadecimal string that can be any and unique. The function of boundary is used to segment fields and separate different parameter parts.
The multipart/from-data specification is defined in rfc2388. For details on fields, see the specification.
--boundary
Content-Disposition: form-data; name="Parameter name"Boundary Content-disposition :form-data; Name = "Form control name"; Filename = "upload filename" content-type :mime Type binary data of the file to be uploaded --boundary--Copy the codeSplicing uploaded files can be basically divided into the following three parts: upload parameters, upload information, and upload files. And through utF-8 format encoding, the server also uses the same decoding method, you can obtain uploaded files and information. Note that the number of newline characters is fixed, which is a fixed protocol format. It should not be too much or too little, which will cause server parsing failure.
- (NSData *)writeMultipartFormData:(NSData *)data
parameters:(NSDictionary *)parameters {
if (data.length == 0) {
return nil;
}
NSMutableData *formData = [NSMutableData data];
NSData *lineData = [@"\r\n" dataUsingEncoding:NSUTF8StringEncoding];
NSData *boundary = [kBoundary dataUsingEncoding:NSUTF8StringEncoding];
// Concatenate upload parameters
[parameters enumerateKeysAndObjectsUsingBlock:^(id key, id obj, BOOL *stop) {
[formData appendData:boundary];
[formData appendData:lineData];
NSString *thisFieldString = [NSString stringWithFormat:@"Content-Disposition: form-data; name=\"%@\"\r\n\r\n%@", key, obj];
[formData appendData:[thisFieldString dataUsingEncoding:NSUTF8StringEncoding]];
[formData appendData:lineData];
}];
// Splice the uploaded information
[formData appendData:boundary];
[formData appendData:lineData];
NSString *thisFieldString = [NSString stringWithFormat:@"Content-Disposition: form-data; name=\"%@\"; filename=\"%@\"\r\nContent-Type: %@".@"name".@"filename".@"mimetype"];
[formData appendData:[thisFieldString dataUsingEncoding:NSUTF8StringEncoding]];
[formData appendData:lineData];
[formData appendData:lineData];
// Splice the uploaded file
[formData appendData:data];
[formData appendData:lineData];
[formData appendData: [[NSString stringWithFormat:@"--%@--\r\n", kBoundary] dataUsingEncoding:NSUTF8StringEncoding]].return formData;
}
Copy the codeThe content-type and Content-Length headers must also be set for form submission, otherwise the request will fail. Content-length is not mandatory, depending on the backend support.
When setting the request header, be sure to add boundary, which should be the same as the boundary used for stitching uploaded files. The server takes boundary from the request header to parse the uploaded file.
NSString *headerField = [NSString stringWithFormat:@"multipart/form-data; charset=utf-8; boundary=%@", kBoundary];
[request setValue:headerField forHTTPHeaderField:@"Content-Type"];
NSUInteger size = [[[NSFileManager defaultManager] attributesOfItemAtPath:uploadPath error:nil] fileSize];
headerField = [NSString stringWithFormat:@"%lu", size];
[request setValue:headerField forHTTPHeaderField:@"Content-Length"];
Copy the codeWe then create NSURLSessionUploadTask with the following code and call Resume to initiate the request, implementing the corresponding proxy callback.
// Initiate a network request
NSURLSessionUploadTask *uploadTask = [self.backgroundSession uploadTaskWithRequest:request fromData:fromData];
[uploadTask resume];
// called after the request completes, whether it succeeds or fails
- (void)URLSession:(NSURLSession *)session
task:(NSURLSessionTask *)task
didCompleteWithError:(NSError *)error {
}
// Update upload progress, callback multiple times
- (void)URLSession:(NSURLSession *)session
task:(NSURLSessionTask *)task
didSendBodyData:(int64_t)bytesSent
totalBytesSent:(int64_t)totalBytesSent
totalBytesExpectedToSend:(int64_t)totalBytesExpectedToSend {
}
// Data receive completes callback
- (void)URLSession:(NSURLSession *)session
dataTask:(NSURLSessionDataTask *)dataTask
didReceiveData:(NSData *)data {
}
// Process background upload tasks. This method will be called back after the upload task of the current session ends.
- (void)URLSessionDidFinishEventsForBackgroundURLSession:(NSURLSession *)session {
}
Copy the codeHowever, if you think this completes an upload function, too young too simple~
The background to upload
Background uploads are not supported for fromData uploads. If you want to implement background upload, you need to upload files fromFile mode. Not only that, fromData has other pits.
- (NSURLSessionUploadTask *)uploadTaskWithRequest:(NSURLRequest *)request fromData:(NSData *)bodyData;
- (NSURLSessionUploadTask *)uploadTaskWithRequest:(NSURLRequest *)request fromFile:(NSURL *)fileURL;
Copy the codeMemory footprint
We found that by uploading files fromData:, the memory went up and never went down, either using NSURLSession or AFNetworking. Small files are fine, it’s not obvious, but if it’s a few hundred megabytes it’s obvious that there’s a memory peak, and it goes up and doesn’t go down. WTF?
There are two types of uploads, but if we change fromData: uploads to fromFile: uploads, we can fix the memory problem. So, we can think of the fromData: upload method as the imageNamed method of UIImage, and then the NSData file is stored in memory, it’s not recycled. FromFile: loads files locally and can be recycled when the files are uploaded. And if you want to support background uploads, you have to upload fromFile:.
OK, so if we find the problem, we’ll do it and change the upload logic to fromFile:.
// Write the shard locally
NSString *filePath = [NSString stringWithFormat:@"%@/%ld"[self segmentDocumentPath], currentIndex];
BOOL write = [formData writeToFile:filePath atomically:YES];
// Create a shard folder
- (NSString *)segmentDocumentPath {
NSString *documentName = [fileName md5String];
NSString *filePath = [[SVPUploadCompressor compressorPath] stringByAppendingPathComponent:documentName];
BOOL needCreateDirectory = YES;
BOOL isDirectory = NO;
if ([[NSFileManager defaultManager] fileExistsAtPath:filePath isDirectory:&isDirectory]) {
if (isDirectory) {
needCreateDirectory = NO;
} else{[[NSFileManager defaultManager] removeItemAtPath:filePath error:nil]; }}if (needCreateDirectory) {
[[NSFileManager defaultManager] createDirectoryAtPath:filePath
withIntermediateDirectories:YES
attributes:nil
error:nil];
}
return filePath;
}
Copy the codeSince you’re passing a local shard path in via the fromFile: method, you need to pre-shard the file and save it locally. At the same time, boundary information needs to be spliced.
Therefore, before the upload task starts, we fragment the file and splice the information, and then write the fragment file to the local. To facilitate management, MD5 is performed to create shard folders based on unique file names. Shard files are named by subscripts and written locally. After the file is uploaded, delete the entire folder. Of course, all of these file operations are done in an asynchronous thread, so as not to affect the UI thread.
We tested the upload with a 400MB video, and as we can see from the image above, the red circle is the time we uploaded the file. Switching from fromFile to fromFile, the peak uploaded files will hover around 10MB, which is fairly friendly for older, low-memory devices like the iPhone6, and won’t cause low-end devices to crash or stall.
Dynamic fragmentation
There are many network conditions when users upload, such as WiFi, 4G, weak network and so on. If the upload fragment is too large, the failure rate may increase. If the upload fragment is too small, too many network requests will be generated, resulting in too many useless resources such as boundary, header and data link waste.
To solve this problem, we adopt a dynamic shard size strategy. According to the specific calculation strategy, the upload speed of the first shard is used as the speed shard in advance, and the size of the speed shard is fixed. According to the result of speed measurement, other fragment sizes are dynamically sharded to ensure that the fragment size can maximize the current network speed.
if ([Reachability reachableViaWiFi]) {
self.segmentSize = 500 * 1024;
} else if ([Reachability reachableViaWWAN]) {
self.segmentSize = 300 * 1024;
}
Copy the codeOf course, if this sharding method is too complicated, you can also adopt a castrated version of dynamic sharding strategy. That is, a fragment size is fixed if it is WiFi, and a fragment size is fixed if it is traffic. However, this strategy is not stable, as many phones have faster Internet speeds than WiFi, and we can’t guarantee that WiFi will be 100 megabit fiber.
Parallel to upload
If all uploaded tasks use the same NSURLSession, they can remain connected, saving the cost of establishing and disconnecting. On iOS platform, NSURLSession supports 4 connections to a Host, so if we adopt parallel upload, we can make better use of the current network.
The number of parallel upload not more than 4 on iOS, maximum number of connections can be through NSURLSessionConfiguration Settings, and the number is best not to write to death. Similarly, the maximum number of connections should be calculated at the beginning of the upload task based on the current network environment and set to Configuration.
Through our online user data analysis, the online environment uses the way of parallel task upload, and the upload speed is about four times higher than that of serial upload. The calculation is the size of file uploads per second.
IPhone serial upload:715KB /s iPhone parallel upload:2909 kb/s
Copy the codeThe queue manager
The fragment upload may fail due to network speed. Failed tasks should be managed by a separate queue and retransmitted when appropriate.
For example, if a 500MB file is fragmented and each slice is 300KB, more than 1700 fragments are generated. Each fragment file corresponds to an upload task. If 1700 uploadTasks are created in one go while uploading, NSURLSession can handle this without causing a large memory spike. But I don’t think that’s a good idea because there aren’t actually so many requests going out at once.
// the array of fragments has been uploaded successfully
@property (nonatomic.strong) NSMutableArray *successSegments;
/// An array of queues to upload
@property (nonatomic.strong) NSMutableArray *uploadSegments;
Copy the codeSo when I created the upload task, I set a maximum number of tasks so that no more than this number of requests can be made to NSURLSession at the same time. Note that the maximum number of tasks is the number of tasks THAT I created uploadTask, not the maximum number of concurrent tasks. The maximum number of concurrent tasks is controlled by NSURLSession, so I don’t interfere.
I place all the tasks to be uploaded in uploadSegments. After successful uploads, I extract one or more tasks from the array and ensure that the number of tasks to be uploaded at the same time does not exceed the maximum number. The failed tasks theoretically also need to wait for uploading, so I insert the failed tasks into uploadSegments at the bottom of the queue. This ensures that failed tasks will continue to be retried after uploading tasks are completed.
Successful tasks are placed in successSegments and always have no intersection with uploadSegments. The two queues do not store uploadTask, but the index of the shard, which is why I named the shard with the index. When successSegments equals the number of fragments, all tasks are uploaded.
File download
NSURLSession is run in a separate process, so network requests initiated through this type are run independently of the application. Even if the App suspends or kills the request, the request will not be stopped. When downloading a task, the task will continue even if the App is killed, allowing the next startup to use the download result or continue downloading.
Creating a downloadTask is as simple as uploading code. Create a downloadTask using NSURLSession and call resume to start a downloadTask.
NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
NSURLSession *session = [NSURLSession sessionWithConfiguration:config
delegate:self
delegateQueue:[NSOperationQueue mainQueue]];
NSURL *url = [NSURL URLWithString:@"http://vfx.mtime.cn/Video/2017/03/31/mp4/170331093811717750.mp4"];
NSURLRequest *request = [[NSURLRequest alloc] initWithURL:url];
NSURLSessionDownloadTask *downloadTask = [session downloadTaskWithRequest:request];
[downloadTask resume];
Copy the codeSuspend the download task by calling suspend and resume. Suspend and resume need to be paired. The default value is 60 seconds. If a timeout occurs, the system disconnects the TCP connection, and you cannot call resume again. Can be set by timeoutIntervalForResource NSURLSessionConfiguration upload and download resources time-consuming. Suspend is only for download tasks; other tasks are suspended and restart.
The following two methods are more basic for downloading. They are used to receive the download progress and the address of the temporary file after downloading. DidFinishDownloadingToURL: method is required, when after the download download file is written to a temporary file under the Library/Caches, we need to move this file to your own directory, the temporary directory at a time will be deleted in the future.
// Receive data from the server, download the progress callback
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask
didWriteData:(int64_t)bytesWritten
totalBytesWritten:(int64_t)totalBytesWritten
totalBytesExpectedToWrite:(int64_t)totalBytesExpectedToWrite {
CGFloat progress = (CGFloat)totalBytesWritten / (CGFloat)totalBytesExpectedToWrite;
self.progressView.progress = progress;
}
// Call back after downloading
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask
didFinishDownloadingToURL:(NSURL *)location {
}
Copy the codeBreakpoint continuingly
The HTTP protocol supports the resumable operation of the breakpoint. The Range field is set in the request header at the beginning of the download request, indicating where to start the download.
Range:bytes=512000-
Copy the codeAfter receiving the request from the client, the server starts to transfer data in the position of 512kb and uses the content-range field to inform the client of the starting position of data transfer.
Content-Range:bytes 512000-/1024000
Copy the codeDownloadTask task start request, you can call cancelByProducingResumeData: method can cancel the download, and can obtain a resumeData resumeData some breakpoints in the download information. You can write resumeData locally and then use this file for breakpoint continuation.
NSString *library = NSSearchPathForDirectoriesInDomains(NSLibraryDirectory.NSUserDomainMask.YES).firstObject;
NSString *resumePath = [library stringByAppendingPathComponent:[self.downloadURL md5String]];
[self.downloadTask cancelByProducingResumeData:^(NSData * _Nullable resumeData) {
[resumeData writeToFile:resumePath atomically:YES];
}];
Copy the codeBefore creating download task, can judge the current task is to restore the task, if there is one call downloadTaskWithResumeData: method and introduced into a resumeData, can restore before download, and recreate a downloadTask task.
NSString *library = NSSearchPathForDirectoriesInDomains(NSLibraryDirectory.NSUserDomainMask.YES).firstObject;
NSString *resumePath = [library stringByAppendingPathComponent:[self.downloadURL md5String]];
NSData *resumeData = [[NSData alloc] initWithContentsOfFile:resumePath];
self.downloadTask = [self.session downloadTaskWithResumeData:resumeData];
[self.downloadTask resume];
Copy the codeTasks suspended by suspend and Resume are the same object downloadTask, while tasks resumed by Cancel and then resumeData create a new downloadTask task.
When calling downloadTaskWithResumeData: recovering after download, can under the callback methods. The fileOffset callback parameter is the last file download size, and the expectedTotalBytes parameter is the estimated total file size.
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask
didResumeAtOffset:(int64_t)fileOffset
expectedTotalBytes:(int64_t)expectedTotalBytes;
Copy the codeThe background to download
Through backgroundSessionConfigurationWithIdentifier upload or download the background type NSURLSessionConfiguration method to create the background, and set up a unique identifier, You need to ensure that this identity is unique between sessions. Background tasks support only HTTP and HTTPS tasks, but not tasks of other protocols.
NSURLSessionConfiguration *config = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:@"identifier"];
[NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:[NSOperationQueue mainQueue]];
Copy the codeBy NSURLSession backgroundSessionConfigurationWithIdentifier method to create the request task will be conducted in the process of system separate, so even if the App process is kill also is not affected, still can continue to perform the requested task. If the application is system kill, starting next time and perform didFinishLaunchingWithOptions can be created by the same identifier NSURLSession and NSURLSessionConfiguration, The system associates the newly created NSURLSession with the NSURLSession that is running in a separate process.
Start the program and perform didFinishLaunchingWithOptions method, according to the following method to create NSURLSession can will create the Session and the Session before the new binding, and automatically start download task before implementation. The NSURLSession proxy method is resumed after the previous task is restored, and subsequent tasks are performed.
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
NSURLSessionConfiguration *config = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:@"identifier"];
[NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:[NSOperationQueue mainQueue]];
return YES;
}
Copy the codeWhen the application is in the Background, the download can continue. If the client has not enabled Background Mode, the client progress will not be called back. The next time you enter the foreground, you will continue to call back the new progress.
If the download is complete in the background, the UI is refreshed by notifying the application through the AppDelegate callback method. Because the download is done in a separate process, the downloaded callback is invoked even if the business layer code stops executing. Allow the user to process the business logic and refresh the UI during callbacks.
After calling this method, you can start refreshing the UI, and calling completionHandler means the refresh is over, so the upper level business has some control logic to do. DidFinishDownloadingToURL to call opportunity than this method, later still in that method can judge the downloaded file. Since there may be multiple download tasks in a project, you need identifiers to distinguish the download tasks.
- (void)application:(UIApplication *)application handleEventsForBackgroundURLSession:(NSString *)identifier completionHandler:(void(^) (void))completionHandler {
ViewController *vc = (ViewController *)self.window.rootViewController;
vc.completionHandler = completionHandler;
}
Copy the codeNote that if there are more than one Identifier task with the same name, the session created will continue to execute all tasks with the same name. NSURLSessionConfiguration also provides the following attributes, whether to start the App when the session download task to complete, the default value is YES, if set to NO background download will be affected.
@property BOOL sessionSendsLaunchEvents;
Copy the codeA series of proxy method calls are designed during the background download, in the following order.
Video file download
Now many video apps have video download function, video download is certainly not a simple download down an MP4 can, here is the video download related knowledge.
- The video address is usually obtained from the server, so you need to request the interface to obtain the download address first. This address can be requested by an interface, or it can be concatenated in a fixed format.
- Now there are many video apps with streaming free services, such as Tencent King card, Ant Bao card and so on. The essence of streaming free services is right
m3u8,ts,mp4Address package a layer, request data directly to the address given by the operator, the operator to do a transfer operation. - In order to stream video
m3u8For example, with a no-stream address, download firstm3u8File. This file is usually encrypted and the client will correct it after downloadingm3u8A filedecodeAnd get the realm3u8File. m3u8The file is essentiallytsA collection of clips, video playback ortsFragment. Later onm3u8The file is parsed and obtainedtsFragment address and willtsThe download addresses can be downloaded one by one or in parallel after they are converted to stream-free addresses.m3u8The downloaded files are stored in a fixed format in a folder corresponding to the cached videos.tsSlice names are named after numbers, for example0.tsThe subscripts start at 0.- all
tsAfter downloading the fragment, generate it locallym3u8File. m3u8There are two types of files: remote file and local file. The remote file is the normal download address, local filem3u8The file is passed in while the local video is playing. Format and commonm3u8The files are similar. The difference istsAn address is a local address, such as the one below.
#EXTM3U
#EXT-X-TARGETDURATION:30
#EXT-X-VERSION:3
# EXTINF: 9.28.
0.ts
# EXTINF: 33.04.
1.ts
# EXTINF: 30.159.
2.ts
# EXTINF: 23.841.
3.ts
#EXT-X-ENDLIST
Copy the codeM3u8 file
Http Live Streaming (HLS) is a Streaming media protocol introduced by Apple. It consists of two parts, M3U8 files and TS files. Ts files are used because multiple Ts can be seamlessly spliced together and a single TS can be played individually. However, due to the mp4 format, the split MP4 file will cause the image to tear or audio loss when played alone. If multiple MP4 files are downloaded separately, playing will cause discontinuity problems.
M3u8 is the Unicode version of M3U, a video format launched by Apple. It is a streaming media transfer protocol based on HTTP. The M3U8 protocol cuts a media file into multiple small files and uses HTTP protocol for data transmission. The resource server path where the small files reside is stored in the. M3u8 file. After receiving the M3U8 file, the client can download different files according to the path of resource files in the file.
The M3U8 file must be encoded in UTF-8 format, with tags beginning with #EXT in the file and case sensitive. Any other string beginning with # is considered a comment. M3u8 can be divided into vod and live broadcast. After the first request of the. M3u8 file, the downloaded TS clips can be played sequentially. Live streaming requires an incremental download of the.m3u8 file over time, and subsequent ts files.
There are many tags in M3U8. Here are some of the tags or main tags used in the project. It is easy to slice mp4 or FLV files with ffmpeg command.
- Start tag, which must be at the beginning of the entire file.
#EXTM3U
- End tag, which must be at the end of the entire file.
#EXT-X-ENDLIST
- The current file version, if not specified, is 1 by default
#EXT-X-VERSION
- all
tsMaximum length of a segment.
#EXT-X-TARGETDURATION
- The current
tsSegment length.
#EXTINF
If it does not start with #EXT or #, it is usually a ts fragment download address. A path can be an absolute path or a relative path, and the absolute path is used in our project. But the amount of data relative to the path will be relatively small, but the people watching the video won’t have too bad Internet speeds.
The following is the relative path address. There is only segment1.ts in the file, which represents the path relative to m3u8, which is the following path.
https://data.vod.itc.cn/m3u8
https://data.vod.itc.cn/segment1.ts
Copy the codeCommon mistakes
A background URLSession with identifier backgroundSession already exists
Copy the codeThis error is displayed if you repeat an existing download task in the background. Call the finishTasksAndInvalidate method to invalidate the task when the page or program exits.
[[NSNotificationCenter defaultCenter] addObserver:self
selector:@selector(willTerminateNotification)
name:UIApplicationWillTerminateNotification
object:nil];
- (void)willTerminateNotification {
[self.session getAllTasksWithCompletionHandler:^(NSArray<__kindof NSURLSessionTask *> * _Nonnull tasks) {
if (tasks.count) {
[self.session finishTasksAndInvalidate]; }}]; }Copy the code