preface

I have been busy with work and life, so I haven’t updated my article for a month. Some time ago, I collected a lot of Kindle books. When I was writing crawler, I wanted to print the real-time downloading progress of books resources on the console for the convenience of observing the progress. Today, I will talk about how to realize this small function.

The body of the

First of all, when we request Requests for a resource path, the response to that request will be downloaded back immediately, so we can’t get progress. But Requests provides us with a stream to get progress in real time. For example, download IDEA:

Request_url = 'https://download.jetbrains.com/idea/ideaIU-2018.2.1.exe' res = requests. Get (request_url, stream = True)Copy the code

When the above statement is executed, only the response header is downloaded and returned to us, so we can get the data we need, such as content-length:

content_length = res.headers['content-length']Copy the code

We then use response. iter_content to control the workflow to iterate over the resource data; In addition, in Python3 adding \r at the beginning of the print causes the cursor to return to the first line, without newline, which gives the effect of a progress bar, so the final code reads:

import requests from contextlib import closing if __name__ == '__main__': Url = 'https://download.jetbrains.com/idea/ideaIU-2018.2.1.exe' with closing (requests. Get (url, stream=True)) as response: Chunk_size = 1024 # Content_size = int(response.headers['content-length']) # data_count = 0 with open('idea.exe', "wb") as file: for data in response.iter_content(chunk_size=chunk_size): Write (data) data_count = data_count + len(data) now_jd = (data_count/content_size) * 100 print("\r %d%%(%d/%d) - %s" % (now_jd, data_count, content_size, url), end=" ")Copy the code

Results as follows:

File download progress: 6% (36305920/540246736) - https://download.jetbrains.com/idea/ideaIU-2018.2.1.exeCopy the code

= > > > >

Afterword.

Life is to grow up in constant learning and tossing, as they practice while taking a rest.