This article is participating in Python Theme Month. See the link for details.

Sometimes, we need to use proxy IP, such as in crawler, but after we get the IP, we may not know how to verify whether the IP is valid. In this case, we can use Python to carry the IP to simulate visiting a website. If we fail to visit it for many times, it indicates that the proxy is invalid. The code is as follows:

import requests
import random
import time

http_ip = [
    '118.163.13.200:8080'.'222.223.182.66:8000'.'51.158.186.242:8811'.'171.37.79.129:9797'.'139.255.123.194:4550'
]

for i in range(10) :try:
        ip_proxy = random.choice(http_ip)
        proxy_ip = {
            'http': ip_proxy,
            'https': ip_proxy,
        }
        print('Using proxy IP:', proxy_ip)
        response = requests.get("http://httpbin.org/ip", proxies=proxy_ip).text
        print(response)
        print('Current IP is valid')
        time.sleep(2)
    except Exception as e:
        print(e.args[0])
        print('Current IP is invalid')
        continue

Copy the code

The running results are as follows:

Use proxy IP: {'http': '118.163.13.200:8080'.'https': '118.163.13.200:8080'}
HTTPConnectionPool(host='118.163.13.200', port=8080): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('
      
       : Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connected party did not reply properly after a period of time or the connected host did not respond. '
      ))) Current IP is invalid using proxy IP: {'http': '51.158.186.242:8811'.'https': '51.158.186.242:8811'}
{
  "origin": "51.158.186.242"} current IP IP address of the proxy in use: {'http': '222.223.182.66:8000'.'https': '222.223.182.66:8000'}
{
  "origin": "139.202.62.84, 222.223.182.66"} current IP IP address of the proxy in use: {'http': '51.158.186.242:8811'.'https': '51.158.186.242:8811'}
{
  "origin": "51.158.186.242"} current IP IP address of the proxy in use: {'http': '51.158.186.242:8811'.'https': '51.158.186.242:8811'}
{
  "origin": "51.158.186.242"} current IP IP address of the proxy in use: {'http': '222.223.182.66:8000'.'https': '222.223.182.66:8000'}
HTTPConnectionPool(host='222.223.182.66', port=8000): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('
      
       : Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connected party did not reply properly after a period of time or the connected host did not respond. '
      ))) Current IP is invalid using proxy IP: {'http': '139.255.123.194:4550'.'https': '139.255.123.194:4550'}
HTTPConnectionPool(host='139.255.123.194', port=4550): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('
      
       : Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connected party did not reply properly after a period of time or the connected host did not respond. '
      ))) Current IP is invalid using proxy IP: {'http': '51.158.186.242:8811'.'https': '51.158.186.242:8811'}
{
  "origin": "51.158.186.242"} current IP IP address of the proxy in use: {'http': '51.158.186.242:8811'.'https': '51.158.186.242:8811'}
{
  "origin": "51.158.186.242"} current IP IP address of the proxy in use: {'http': '222.223.182.66:8000'.'https': '222.223.182.66:8000'}
HTTPConnectionPool(host='222.223.182.66', port=8000): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('
      
       : Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connected party did not reply properly after a period of time or the connected host did not respond. '
      )) The current IP address is invalidCopy the code