The Requests module is a wrapper around URllib, a crawler that masquerades as a browser making a request to a web server to get a response. Instead of downloading the entire web page, the Requests module is a partial crawler that adds headers and parameters to request specific web interfaces.

The Requests module sends HTTP requests to the server for data and supports get and POST protocol requests. In the crawler process, the response information we need can be obtained through the control of query parameters, request and other information. Meanwhile, the POST method supports the acquisition of accurate crawler data through uploading and other methods.

1. GET method: do not add any request parameters

 1# -*- coding: UTF-8 -*-
 2import requests
 3
 4Define the requested URL
 5url = "http://httpbin.org"
 6
 7Use the GET method to initiate a request
 8response = requests.get(url=url)
 9
10Print the response status. 200 indicates success
11print "Response status",response.status_code
12Print web page text
13print "Webpage text:",response.text
Copy the code

2, GET method: add request parameters

 1# -*- coding: UTF-8 -*-
 2import requests
 3
 4# define the requested URL interface (the interface that needs to be requested), assuming the interface is http://httpbin.org/get
 5url = "http://httpbin.org/get"
 6
 7Define request header information (dictionary type)
 8headers = {
 9    "Accept": "* / *".10    "Accept-Encoding": "gzip".11    "User-Agent": "".12}
13
14Define request parameter information (dictionary type)
15params = {'name': 'Python Concentration Camp '.'type': 'python'}
16
17Use GET to make a request (add headers, add parameters)
18response = requests.get(url=url,headers=headers,params=params)
19
20Print the response status. 200 indicates success
21print "Response status",response.status_code
22Print response information
23print "Response message:",response.content
Copy the code

3. POST method: submit ordinary data

 1# -*- coding: UTF-8 -*-
 2import requests
 3
 4# define the requested URL interface (the interface that needs to be requested), assuming the interface is http://httpbin.org/post
 5url = "http://httpbin.org/post"
 6
 7Define request header information (dictionary type)
 8headers = {
 9    "Accept": "* / *".10    "Accept-Encoding": "gzip".11    "User-Agent": "".12    Define request data format, JSON
13    "Content-Type": "application/json".14}
15
16Define request parameter information (dictionary type)
17params = {'name': U 'Python Concentration Camp '.'type': 'python'}
18
19Add header information, add parameter information.
20# set data = params
21response = requests.post(url=url,headers=headers,data=params)
22
23Print the response status. 200 indicates success
24print "Response status",response.status_code
25Print response information
26print "Response message:",response.content
Copy the code

4. POST method: Submit file data

 1# file parameter (dictionary type)
 2files = {
 3    'file': (
 4        # file name
 5        'python.png'.6        # File path, open file
 7        open('C:/python.png'.'rb'),
 8        # File type, image
 9        'image/png'),
10}
11
12Add header information, add parameter information.
13Data = params(defined parameter)
14# set file parameters files = files(defined files)
15response = requests.post(url=url, headers=headers, data=params, files=files)
16Print the response status. 200 indicates success
17print "Response status", response.status_code
18Print response information
19print "Response message:", response.content
Copy the code