This is the seventh day of my participation in the August More text Challenge. For details, see: August More Text Challenge

Introduction to the

Requests is an excellent HTTP client library implemented in Python, and the Requests module is perfectly suited to the needs of the web today. “Requests allow you to send pure HTTP/1.1 Requests with no additional action.” This allows us to use, can be very simple and efficient.

features

  • Keep-alive & connection pool
  • Internationalize domain names and URLs
  • Persistent session with cookies
  • Browser-based SSL authentication
  • Automatic content decoding
  • Basic/summary authentication
  • Elegant key/value cookies
  • Automatically decompressed
  • Unicode response body
  • Support for HTTP(S) proxies
  • Upload files in blocks
  • Flow to download
  • Connection timeout
  • Block the request
  • .netrc-enabled

Sending network Requests using Requests is very simple. You just need to import the Requests module first, and then try to make a GET request for the page using the Requests. Get (” url “) function. This function returns a Response object from which we can get all the information we want. Of course, there are other types of HTTP requests that can be sent, such as POST, PUT, DELETE, or HEAD.

Because of the simplicity and power of the Requests library, it is currently being heavily used by developers for both client side Requests and crawler development. While Requests are excellent, there is a problem with garbled Chinese characters that need to be addressed if they are not used properly.

Result = request.get (" http://www.baidu.com ") result_text = result.textCopy the code

When you get the content of the page using the above statement, it is likely that garbled characters will appear because result.text returns Unicode data. There are two solutions: 1. Get bytes data and convert it to a string

result = requests.get(url)
data = result.content
result_text = str(data, 'utf-8')
Copy the code

2.Requests automatically decode content from the server. If you don’t specify, Requests will make an educated guess about the encoding of the response based on the HTTP header. When you access R. ext, Requests use their inferred text encoding. But such speculations are sometimes inaccurate. It’s even possible to use the self-encoding of the Requests library — “ISO-8859-1” — so we can specify the text encoding before using result.text to get the content

Result = request.get (" http://www.baidu.com ") result.encoding=' utF-8 'result_text = result.textCopy the code

Green destiny both