Download should be a required function of every App, if we don’t use a third party framework, we need to implement the download tool ourselves. What can we do if we implement it ourselves?
First, if the server file supports resumable breakpoints, the main function points we need to implement are as follows:
- Multithreading, breakpoint continuation download
- Download management: Start, pause, continue, cancel, restart
If the server file does not support breakpoint continuation, normal single-threaded downloads can only be performed and cannot be paused or continued. Of course, the general case of server files should support breakpoint continuation bar!
Below are the single task download, multi-task list download, and Service download:
Basic implementation principle:
Since our download is based on OKHTTP, we first need an OkHttpManager class that encapsulates the most basic network requests:
public class OkHttpManager {... Omit.../** * asynchronous (according to breakpoint request) **@param url
* @param start
* @param end
* @param callback
* @return* /
public Call initRequest(String url, long start, long end, final Callback callback) {
Request request = new Request.Builder()
.url(url)
.header("Range"."bytes=" + start + "-" + end)
.build();
Call call = builder.build().newCall(request);
call.enqueue(callback);
return call;
}
/** * Synchronize request **@param url
* @return
* @throws IOException
*/
public Response initRequest(String url) throws IOException {
Request request = new Request.Builder()
.url(url)
.header("Range"."bytes=0-")
.build();
return builder.build().newCall(request).execute();
}
/** * If the server file has been changed **@param url
* @param lastModify
* @return
* @throws IOException
*/
public Response initRequest(String url, String lastModify) throws IOException {
Request request = new Request.Builder()
.url(url)
.header("Range"."bytes=0-")
.header("If-Range", lastModify)
.build();
return builder.build().newCall(request).execute();
}
/** * Initializes the certificate on HTTPS requests **@param certificates
* @return* /
public void setCertificates(InputStream... certificates) {
try {
CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509");
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
keyStore.load(null);
int index = 0;
for (InputStream certificate : certificates) {
String certificateAlias = Integer.toString(index++);
keyStore.setCertificateEntry(certificateAlias, certificateFactory.generateCertificate(certificate));
try {
if(certificate ! =null)
certificate.close();
} catch (IOException e) {
}
}
SSLContext sslContext = SSLContext.getInstance("TLS");
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(keyStore);
sslContext.init(null, trustManagerFactory.getTrustManagers(), new SecureRandom());
builder.sslSocketFactory(sslContext.getSocketFactory());
} catch(Exception e) { e.printStackTrace(); }}}Copy the codeThis class contains basic timeout configuration, making asynchronous requests based on breakpoint information, verifying server files for updates, HTTPS certificate configuration, and more. So you have the network request part.
Next, we also need database support to record basic information about the downloaded files. Here we use SQLite with only one table:
/** * download_info table builder clause */
public static final String CREATE_DOWNLOAD_INFO = "create table download_info ("+"id integer primary key autoincrement, "+"url text."+"path text."+"name text."+"child_task_count integer."+"current_length integer."+"total_length integer."+"percentage real."+"last_modify text."+"date text)";Copy the codeOf course, there are corresponding table to add, delete, change to check the tool class, specific reference source code.
Due to the need for download management, thread pool is also essential, so as to avoid excessive creation of child threads, to achieve the purpose of reuse, of course, the size of the thread pool can be configured according to the requirements, the main code is as follows:
public class ThreadPool {
// Number of tasks that can be downloaded simultaneously (number of core threads)
private int CORE_POOL_SIZE = 3;
// Cache queue size (maximum number of threads)
private int MAX_POOL_SIZE = 20;
// The timeout period (in seconds) in which non-core threads are idle
private long KEEP_ALIVE = 10L;
private ThreadPoolExecutor THREAD_POOL_EXECUTOR;
private ThreadFactory sThreadFactory = new ThreadFactory() {
private final AtomicInteger mCount = new AtomicInteger();
@Override
public Thread newThread(@NonNull Runnable runnable) {
return new Thread(runnable, "download_task#"+ mCount.getAndIncrement()); }}; . Omit...public void setCorePoolSize(int corePoolSize) {
if (corePoolSize == 0) {
return;
}
CORE_POOL_SIZE = corePoolSize;
}
public void setMaxPoolSize(int maxPoolSize) {
if (maxPoolSize == 0) {
return;
}
MAX_POOL_SIZE = maxPoolSize;
}
public int getCorePoolSize(a) {
return CORE_POOL_SIZE;
}
public int getMaxPoolSize(a) {
return MAX_POOL_SIZE;
}
public ThreadPoolExecutor getThreadPoolExecutor(a) {
if (THREAD_POOL_EXECUTOR == null) {
THREAD_POOL_EXECUTOR = new ThreadPoolExecutor(
CORE_POOL_SIZE, MAX_POOL_SIZE,
KEEP_ALIVE, TimeUnit.SECONDS,
new LinkedBlockingDeque<Runnable>(),
sThreadFactory);
}
returnTHREAD_POOL_EXECUTOR; }}Copy the codeNext up is our core download class FileTask, which implements the Runnable interface so that it can be executed in a thread pool. Let’s look at the logic of the run() method:
@Override
public void run() {
try {
File saveFile = new File(path, name);
File tempFile = new File(path, name + ".temp");
DownloadData data = Db.getInstance(context).getData(url);
if(Utils.isFileExists(saveFile) && Utils.isFileExists(tempFile) && data ! =null) {
Response response = OkHttpManager.getInstance().initRequest(url, data.getLastModify());
if (response! =null && response.isSuccessful() && Utils.isNotServerFileChanged(response)) {
TEMP_FILE_TOTAL_SIZE = EACH_TEMP_SIZE * data.getChildTaskCount();
onStart(data.getTotalLength(), data.getCurrentLength(), "".true);
} else {
prepareRangeFile(response);
}
saveRangeFile();
} else {
Response response = OkHttpManager.getInstance().initRequest(url);
if (response! =null && response.isSuccessful()) {
if (Utils.isSupportRange(response)) {
prepareRangeFile(response);
saveRangeFile();
} else {
saveCommonFile(response); } } } } catch (IOException e) { onError(e.toString()); }}Copy the codeIf the downloaded target file, the temporary file that records the breakpoint, and the database record all exist, check whether the server file is updated. If not, start the download according to the previous record. Otherwise, prepare for the breakpoint download. If the record files do not all exist, check whether breakpoint uploading is supported. If breakpoint uploading is supported, follow the flow of breakpoint uploading. Otherwise, ordinary download is used.
First look at the prepareRangeFile() method, where we prepare for breakpoint continuals:
private void prepareRangeFile(Response response) { ................. Omit... try { File saveFile = Utils.createFile(path, name); File tempFile = Utils.createFile(path, name +".temp");
long fileLength = response.body().contentLength();
onStart(fileLength, 0, Utils.getLastModify(response), true);
Db.getInstance(context).deleteData(url);
Utils.deleteFile(saveFile, tempFile);
saveRandomAccessFile = new RandomAccessFile(saveFile, "rws");
saveRandomAccessFile.setLength(fileLength);
tempRandomAccessFile = new RandomAccessFile(tempFile, "rws");
tempRandomAccessFile.setLength(TEMP_FILE_TOTAL_SIZE);
tempChannel = tempRandomAccessFile.getChannel();
MappedByteBuffer buffer = tempChannel.map(READ_WRITE, 0, TEMP_FILE_TOTAL_SIZE);
long start;
long end;
int eachSize = (int) (fileLength / childTaskCount);
for (int i = 0; i < childTaskCount; i++) {
if (i == childTaskCount - 1) {
start = i * eachSize;
end = fileLength - 1;
} else {
start = i * eachSize;
end = (i + 1) * eachSize - 1; } buffer.putLong(start); buffer.putLong(end); } } catch (Exception e) { onError(e.toString()); } finally { ............. Omit... }}Copy the codeThe first step is to clear the history and create a new object file and temporary file. ChildTaskCount means that the file needs to be downloaded by several subtasks. This gives you the size of the task that each subtask needs to download and the breakpoint information is recorded in the temporary file. File download we use the MappedByteBuffer class, more efficient than RandomAccessFile. Executing the onStart() method at the same time will represent the preparation phase for the download, which will be detailed later.
Now look at the saveRangeFile() method:
private void saveRangeFile() { ................. Omit... for (int i =0; i < childTaskCount; i++) { final int tempI = i; Call call = OkHttpManager.getInstance().initRequest(url, range.start[i], range.end[i], new Callback() { @Override public void onFailure(Call call, IOException e) { onError(e.toString()); } @Override public void onResponse(Call call, Response response) throws IOException { startSaveRangeFile(response, tempI, range, saveFile, tempFile); }}); callList.add(call); }... Omit... }Copy the codeIf the response is successful, the file is segmented by the startSaveRangeFile() method:
private void startSaveRangeFile(Response response, int index, Ranges range, File saveFile, File tempFile) { ................. Omit... try { saveRandomAccessFile = new RandomAccessFile(saveFile,"rws");
saveChannel = saveRandomAccessFile.getChannel();
MappedByteBuffer saveBuffer = saveChannel.map(READ_WRITE, range.start[index], range.end[index] - range.start[index] + 1);
tempRandomAccessFile = new RandomAccessFile(tempFile, "rws");
tempChannel = tempRandomAccessFile.getChannel();
MappedByteBuffer tempBuffer = tempChannel.map(READ_WRITE, 0, TEMP_FILE_TOTAL_SIZE); inputStream = response.body().byteStream(); int len; byte[] buffer = new byte[BUFFER_SIZE]; while ((len = inputStream.read(buffer)) ! =- 1) {
/ / cancel
if (IS_CANCEL) {
handler.sendEmptyMessage(CANCEL);
callList.get(index).cancel();
break;
}
saveBuffer.put(buffer, 0, len);
tempBuffer.putLong(index * EACH_TEMP_SIZE, tempBuffer.getLong(index * EACH_TEMP_SIZE) + len);
onProgress(len);
// Exit the save record
if (IS_DESTROY) {
handler.sendEmptyMessage(DESTROY);
callList.get(index).cancel();
break;
}
/ / pause
if(IS_PAUSE) { handler.sendEmptyMessage(PAUSE); callList.get(index).cancel(); break; } } addCount(); } catch (Exception e) { onError(e.toString()); } finally { ................. Omit... }Copy the codeWrite the current file and save the current downloaded location to a temporary file in a while loop:
saveBuffer.put(buffer, 0.len);
tempBuffer.putLong(index * EACH_TEMP_SIZE, tempBuffer.getLong(index * EACH_TEMP_SIZE) + len);Copy the codeThe onProgress() method is also called to send the progress, where cancel, exit save record, and pause need to break the while loop.
Since the download is done in the child thread, we typically need to update the UI in the UI thread based on the download status, so we send the download status data to the UI thread via Handler: that is, calling hander.sendemptyMessage ().
Finally, the FileTask class has a saveCommonFile() method that does a normal download that does not support resumable breakpoints.
The ProgressHandler class sends state data to the UI thread using a Handler.
private Handler mHandler = new Handler() {
@Override
public void handleMessage(Message msg) {
super.handleMessage(msg);
switch (mCurrentState) {
case START:
break;
case PROGRESS:
break;
case CANCEL:
break;
case PAUSE:
break;
case FINISH:
break;
case DESTROY:
break;
case ERROR:
break; }}};Copy the codeIn the handleMessage() method, we do something based on the current download state. If it is START, you need to insert the downloaded data into the database, perform initialization callback, etc. If PROGRESS is performed, the download PROGRESS callback is performed; If CANCEL, delete the target file, temporary file, database record and perform the corresponding callback. If PAUSE is used, update database file records and perform paused callback, etc. If FINISH, delete temporary files and database records and execute the completed callback; DESTROY means download directly from the Activity, and exit the Activity to update the database record. The last ERROR corresponds to the ERROR case. Refer to the source code for more details.
Finally, use the thread pool in the DownloadManger class to perform the download:
ThreadPool.getInstance(a).getThreadPoolExecutor(a).execute(fileTask);
// If the number of tasks being downloaded equals the number of core threads in the thread pool, the newly added task is in the wait state
if (ThreadPool.getInstance().getThreadPoolExecutor().getActiveCount() == ThreadPool.getInstance().getCorePoolSize()) {
downloadCallback.onWait(a); }Copy the codeAnd determine whether the newly added task is in the waiting state, which is convenient for processing in the UI layer. Here the core implementation principle is over, more details can refer to the source code.
How to use:
DownloadManger is a singleton class that contains specific usage operations. We can start, pause, continue, cancel, restart, configure thread pools, configure HTTPS certificates, query data records, and obtain data in the current download state according to the URL:
- There are three ways to start a download task:
1.Through the DownloadManager classstart(DownloadData downloadData, DownloadCallback downloadCallback)Data can set the URL, save path, file name, and number of subtasks:
2. Execute the setOnDownloadCallback(DownloadData, DownloadCallback) method of the DownloadManager class. Bind data and callback, and execute the start(String URL) method.
3. Chain calls need to be made through the DUtil class: for example
DUtil.init(mContext)
.url(url)
.path(Environment.getExternalStorageDirectory() + "/DUtil/")
.name(name.xxx)
.childTaskCount(3)
.build(a).start(callback);Copy the codeStart () method returns a DownloadManager instances of the class, if you don’t care about the return value, using DownloadManger. GetInstance DownloadManager instance of a class can also be obtained (context), For subsequent operations such as pause, continue, and cancel.
The complete callback can be implemented using the DownloadCallback interface:
new DownloadCallback() {
/ /
@Override
public void onStart(long currentSize, long totalSize, float progress) {
}
/ / download
@Override
public void onProgress(long currentSize, long totalSize, float progress) {
}
/ / pause
@Override
public void onPause() {
}
/ / cancel
@Override
public void onCancel() {
}
// Download complete
@Override
public void onFinish(File file) {
}
/ / wait for
@Override
public void onWait() {
}
// Download error
@Override
public void onError(String error) {
}
}Copy the codeYou can also use the SimpleDownloadCallback interface to implement only the required callback methods.
- Pause tasks in download:
pause(String url) -
Resume (String URL) PS: Files that do not support resumable transmission cannot be paused or resumed.
-
Cancel a task: Cancel (String URL) to cancel a task that is downloading or suspended.
- Start downloading again:
restart(String url), pause, downloading, cancelled, completed tasks can restart the download. - Download data save:
Destroy (String url), destroy (String... urls), such as in the Activity directly download, direct exit can be inonDestroy()Method to save data. - Configuring a thread pool:
setTaskPoolSize(int corePoolSize, int maxPoolSize), set the number of core threads and total threads. - Configure the OKHTTP certificate:
setCertificates(InputStream... certificates) - Query individual data in the database
DownloadData getDbData(String url)To query all data:List<DownloadData> getAllDbData()
ps: The database does not save the downloaded data - Get data for a file in the download queue:
DownloadData getCurrentData(String url)
Here the basic introduction is over, more details and specific use are in the demo, unreasonable place also please give advice.
Github address: github.com/Othershe/DU…