In this section, we will introduce the recognition of Sina Weibo gong Verification code. Micro-blog palace verification code is a new type of interactive verification code, each palace will have an indication between the line, indicating the sliding track should be. Verification can be completed only when we slide from the starting grid to the ending grid according to the sliding trajectory, as shown in the picture below.

The track after mouse sliding will be marked by yellow line, as shown in the figure below.

Visit the sina weibo mobile login page, you can see as above authentication code, links to https://passport.weibo.cn/signin/login. The verification code does not appear every time you log in. The verification code appears only when you log in frequently or the account has security risks.

I. Objectives of this section

Our goal is to use the program to identify and pass the verification of the Micro-blog palace verification code.

Second, preparation

In this case, the Python library is Selenium and the Chrome browser is used. Please ensure that the Selenium library, Chrome browser and ChromeDriver have been correctly installed.

Three, identify ideas

Recognition begins with the exploration of laws. The rule is that the four cells of the verification code must have a line, each line will have a corresponding indicating arrow, the shape of the line, including C, Z, X, etc., as shown below.



We find that the same type of wiring tracks are the same, the only difference is the direction of the wiring, as shown in the figure below.


The line trace of the two captcha is the same. However, due to the different indication arrows above the line, the order of sliding palace is different.

To fully recognize sliding grid order, you need to specifically identify the direction of the arrows. The entire captcha arrow faces a total of eight directions, and will appear in different positions. If you want to write an arrow direction recognition algorithm, you need to consider the position of different arrows, find out the pixel coordinates of each position arrow, and calculate the change rule of pixel points, which will become a relatively large workload.

In this case, we can consider using template matching method, which is to save and mark some identified objects in advance, which is called template. Here the captcha image is marked in drag order as a template. Compare the target to be newly identified with each template, and if a matching template is found, the target to be newly identified is successfully identified. Template matching is also a common method in image recognition, which is easy to implement and easy to use.

We must collect enough templates for the template matching method to work well. As for the micro-blog palace verification code, there are only 4 palace verification codes, and the verification code styles are at most 4×3×2×1=24, so we can collect all the templates.

The next thing we need to think about is, which template to use for matching, just arrows or the whole captchas? Let’s weigh the matching accuracy and effort of these two methods.

  • The first is precision. If it is a matching arrow, the target of comparison is an arrow with a range of only a few pixels, so we need to know exactly the pixel of each arrow. Once the pixel is deviated, it will be directly misplaced, leading to a significant discount in the matching result. If we’re matching the whole picture, we don’t have to worry about where the arrows are, and there are wires to help match. Obviously, the accuracy of full graph matching is higher.

  • Then there is the workload. If it is matching the arrow, we need to save all the arrow towards the different templates, and the same position of the arrow towards may differ, the arrow towards the same location may differ, so we need to calculate the location of each arrow and cut out the save as template, one by one, in turn, explore the captcha if there is a matching template corresponding to the position. If it is to match the full picture, we do not need to care about the position and orientation of each arrow, just need to save the full picture of the verification code, and do not need to calculate the position of the arrow during the match. Obviously, it takes less work to match the full graph.

To sum up, we choose the full map matching method for identification. Once the matching template is found, we can get the drag order defined for the template and then simulate the drag.

Get the template

We need to do some preparatory work. Save 24 full captcha images first. Because captchas are random, there are 24 of them. We can write a program to batch save captcha images, and then screen out the needed images, the code is as follows:

import time
from io import BytesIO
from PIL import Image
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

USERNAME = ' '
PASSWORD = ' '

class CrackWeiboSlide():
    def __init__(self):
        self.url = 'https://passport.weibo.cn/signin/login'
        self.browser = webdriver.Chrome()
        self.wait = WebDriverWait(self.browser, 20)
        self.username = USERNAME
        self.password = PASSWORD

    def __del__(self):
        self.browser.close()

    def open(self):
        """Open the page enter the username and password and hit return: None."""
        self.browser.get(self.url)
        username = self.wait.until(EC.presence_of_element_located((By.ID, 'loginName')))
        password = self.wait.until(EC.presence_of_element_located((By.ID, 'loginPassword')))
        submit = self.wait.until(EC.element_to_be_clickable((By.ID, 'loginAction')))
        username.send_keys(self.username)
        password.send_keys(self.password)
        submit.click()

    def get_position(self):
        """Get captcha position :return: Captcha position tuple"""
        try:
            img = self.wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'patt-shadow')))
        except TimeoutException:
            print('Captcha not present')
            self.open()
        time.sleep(2)
        location = img.location
        size = img.size
        top, bottom, left, right = location['y'], location['y'] + size['height'], location['x'], location['x'] + size['width']
        return (top, bottom, left, right)

    def get_screenshot(self):
        """Get webpage screenshot :return: screenshot object"""
        screenshot = self.browser.get_screenshot_as_png()
        screenshot = Image.open(BytesIO(screenshot))
        return screenshot

    def get_image(self, name='captcha.png') :"""Get capTCHA picture: Return: Picture object"""
        top, bottom, left, right = self.get_position()
        print('Captcha location', top, bottom, left, right)
        screenshot = self.get_screenshot()
        captcha = screenshot.crop((left, top, right, bottom))
        captcha.save(name)
        return captcha

    def main(self):
        """Batch fetch captcha :return: Picture object"""
        count = 0
        while True:
            self.open()
            self.get_image(str(count) + '.png')
            count += 1

if __name__ == '__main__':
    crack = CrackWeiboSlide()
    crack.main()Copy the code

You need to change USERNAME and PASSWORD to the USERNAME and PASSWORD of your own microblog. After running for some time, there are many local captcha with numerical names, as shown in the figure below.

Here we just need to pick out 24 different captcha images and name them and save them. The name can be taken directly as the sliding order of the grid, as shown below.

We named the image 4132. PNG, which means the sliding order is 4-1-3-2. According to such rules, we sorted the verification codes into the following 24 pictures, as shown below.

The 24 pictures above are our template. Next, the recognition process simply traverses the template for matching.

Five, template matching

Call get_image() to get the captCHA image object. Then, template matching is performed on the captcha picture object, defining the method as follows:

from os import listdir

def detect_image(self, image):
    """Match image: param image: return: drag order"""
    for template_name in listdir(TEMPLATES_FOLDER):
        print('Matching', template_name)
        template = Image.open(TEMPLATES_FOLDER + template_name)
        if self.same_image(image, template):
            # return order
            numbers = [int(number) for number in list(template_name.split('. ') [0])]print('Drag order', numbers)
            return numbersCopy the code

TEMPLATES_FOLDER is the folder where the template resides. Here we get the file names of all templates using the listdir() method, then iterate over them and compare the captcha to the template using same_image(). If the match is successful, the matched template file name is converted to a list. If template file 3124.png is matched, the result is [3, 1, 2, 4].

The method of comparison is as follows:

def is_pixel_equal(self, image1, image2, x, y):
    ""Param image1: image1: param image2: image2: param x: position x: param y: position y: return: pixels are the same""
    # Take two pixels of the image
    pixel1 = image1.load()[x, y]
    pixel2 = image2.load()[x, y]
    threshold = 20
    if abs(pixel1[0] - pixel2[0]) < threshold and abs(pixel1[1] - pixel2[1]) < threshold and abs(
            pixel1[2] - pixel2[2]) < threshold:
        return True
    else:
        return False

def same_image(self, image, template):
    """Param image: verification code to be identified :param template: template: return:"""
    # Similarity thresholdThreshold = 0.99 count = 0for x in range(image.width):
        for y in range(image.height):
            # check whether the pixels are the same
            if self.is_pixel_equal(image, template, x, y):
                count += 1
    result = float(count) / (image.width * image.height)
    if result > threshold:
        print('Successful match')
        return True
    return FalseCopy the code

Images are also compared using the method of traversing pixels. The same_image() method takes two parameters, image as the captCHA image object to be detected, and template as the template object. Since they are exactly the same size, here we traverse all the pixels of the image. Compare the pixels in the same position. If the pixels are the same, the count is increased by 1. Finally, the ratio of the same pixels to the total pixels is calculated. If the ratio exceeds a certain threshold, the images are judged to be identical and the match is successful. Here, the threshold is set as 0.99, that is, if the similarity ratio of the two is more than 0.99, the match is successful.

Use the above method to match 24 templates in sequence. If the captcha image works, we can always find a matching template, and then we can get the sliding order of the palace.

Six, simulated drag

Next, drag the mouse according to the sliding order to connect each palace, the method is as follows:

def move(self, numbers):
    """Drag by param numbers: :return:"""
    Get four press points
    circles = self.browser.find_elements_by_css_selector('.patt-wrap .patt-circ')
    dx = dy = 0
    for index in range(4):
        circle = circles[numbers[index] - 1]
        # if it is the first loop
        if index == 0:
            # Click on the first click
            ActionChains(self.browser) \
                .move_to_element_with_offset(circle, circle.size['width'] / 2, circle.size['height'] / 2) \
                .click_and_hold().perform()
        else:
            # Small number of moves
            times = 30
            # drag
            for i in range(times):
                ActionChains(self.browser).move_by_offset(dx / times, dy / times).perform()
                time.sleep(1 / times)
        # If it is the last loop
        if index == 3:
            # Release the mouse
            ActionChains(self.browser).release().perform()
        else:
            # Calculate the next offset
            dx = circles[numbers[index + 1] - 1].location['x'] - circle.location['x']
            dy = circles[numbers[index + 1] - 1].location['y'] - circle.location['y']Copy the code

The arguments received by this method are the grid points in order, such as [3,1,2,4]. First, we use the find_elements_by_css_selector() method to get four palace elements, which are in the form of a list, with each element representing a palace. And then you go through all the points in the grid and you do a series of corresponding operations.

If the current traversal is the first palace, then direct mouse click and hold the action, otherwise move to the next palace. Release the mouse if it is currently traversing the last grid, and calculate the offset of moving to the next grid if it is not the last grid.

Through 4 cycles, we can successfully operate the browser to complete the palace verification code drag and fill, release the mouse to identify success. The running effect is shown in the figure below.

The mouse moves slowly from the start position to the end position. When the last grid is released, the identification of the captcha is complete.

At this point, the identification of micro-blog palace verification code is all completed. The verification code window closes automatically. Click the login button to log in to weibo.

Vii. Code of this section

This section of code address is: https://github.com/Python3WebSpider/CrackWeiboSlide.

Eight, epilogue

This section introduces a common template matching image recognition method, simulates the mouse drag action to realize verification code recognition. If we encounter similar captcha, we can use the same idea to identify.


This resource starting in Cui Qingcai personal blog still find: Python3 tutorial | static find web crawler development practical experience

For more crawler information, please follow my personal wechat official account: Attack Coder

Weixin.qq.com/r/5zsjOyvEZ… (Qr code automatic recognition)