29 Crawler project treasure Tutorials that you deserve!

Learn how to directly crack 80% of websites.

This JS decryption to Youdao translation as an example, I believe that you will definitely harvest after reading!

1, web view

2, Youdao translation simple source code

importYou can't have just one user-agent, because Youdao translation has some sort of anti-picker mechanism, so we'll just add headers = {"Accept": "application/json, text/javascript, */*; Q = 0.01"."Accept-Encoding": "gzip, deflate"."Accept-Language": "zh-CN,zh; Q = 0.9"."Connection": "keep-alive"."Content-Length": "244"."Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"."Cookie": "OUTFOX_SEARCH_USER_ID = - 1506602845 - @10.169.0.82; JSESSIONID=aaaUggpd8kfhja1AIJYpx; OUTFOX_SEARCH_USER_ID_NCOO = 108436537.92676207; ___rl__test__cookies=1597502296408"."Host": "fanyi.youdao.com"."Origin": "http://fanyi.youdao.com"."Referer": "http://fanyi.youdao.com/"."user-agent": "Mozilla / 5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36"."X-Requested-With": "XMLHttpRequest"} # params = {"i": "I love you"."from": "UTO"."to": "AUTO"."smartresult": "dict"."client": "fanyideskweb"."salt": "15975022964104"."sign": "7b7db70e0d1a786a43a6905c0daec508"."lts": "1597502296410"."bv": "9ef72dd6d1b2c04a72be6b706029503a"."doctype": "json"."version": "2.1"."keyfrom": "fanyi.web"."action": "FY_BY_REALTlME",
}

url = "http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule"Request. POST (url=url,headers=headers,data=params). Json () print(response)Copy the code

So far, the code of Youdao translation has been completed on the surface, but in fact, there are problems. For example, an error will be reported if the content that needs to be translated is changed below

why

When we re – translate in Youdao Translation, we will initiate new POST requests, and each request will carry different parameter values. If we want to truly implement Youdao Translation, we need to find a way to generate these four parameters, and then use Python to implement the same function

3, JS decryption (details)

Ctrl+Shift+ F to search, type sign

Because encryption must be encrypted at request time we search ==translate_o==

Debug = = = =

Re-enter the translation content in Youdao Translation and stop at the breakpoint

But it turns out the data is already encrypted

But since we are in debug mode, we can go back to the previous step and click the button below

It can be found that salt, sign, LTS and bV are obtained by ==r==, and ==r== is obtained by a function Go into that function

You can see that this is where the encryption is To copy this JS code, we need to use Python code to achieve this

Can see is exactly the same, here is a little bit of a small success 😁

Notice that the bV is different depending on the translation

Next, look at LTS, implemented in Python

Timestamps generated by Python are different from those generated by JS timestamps, but anyone who knows how to make python timestamps look like JS timestamps

That’s ok

Multiply it by 1000 and you get an int and you get rid of the decimal place and you get a string, because JS is a string

So let’s look at I and implement it in Python

The last

Tips:It may be changed in the Youdao Translation update, but this is not a problem, if you look carefully at JS decryption, you can still find the new one

4, Python to achieve JS decryption after the complete code

import requests
from hashlib import md5
import time
importRandom # request url ="http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule"

appVersion = "5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36"

headers = {
    "Accept": "application/json, text/javascript, */*; Q = 0.01"."Accept-Encoding": "gzip, deflate"."Accept-Language": "zh-CN,zh; Q = 0.9"."Connection": "keep-alive"."Content-Length": "244"."Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"."Cookie": "OUTFOX_SEARCH_USER_ID = - 1506602845 - @10.169.0.82; JSESSIONID=aaaUggpd8kfhja1AIJYpx; OUTFOX_SEARCH_USER_ID_NCOO = 108436537.92676207; ___rl__test__cookies=1597502296408"."Host": "fanyi.youdao.com"."Origin": "http://fanyi.youdao.com"."Referer": "http://fanyi.youdao.com/"."user-agent": appVersion,
    "X-Requested-With": "XMLHttpRequest",}def r(e):
    # bv
    t = md5(appVersion.encode()).hexdigest()

    # lts
    r = str(int(time.time() * 1000))

    # i
    i = r + str(random.randint(0.9))

    return {
        "ts": r,
        "bv": t,
        "salt": i,
        "sign": md5(("fanyideskweb" + e + i + "]BjuETDhU)zqSxf-=B#7m").encode()).hexdigest()
    }


def fanyi(word):

    data = r(word)
    params = {
        "i": word,
        "from": "UTO"."to": "AUTO"."smartresult": "dict"."client": "fanyideskweb"."salt": data["salt"]."sign": data["sign"]."lts": data["ts"]."bv": data["bv"]."doctype": "json"."version": "2.1"."keyfrom": "fanyi.web"."action": "FY_BY_REALTlME"} response = requests. Post (url=url,headers=headers,data=paramsreturn response.json()



if __name__ == "__main__":
    while True:
        word = input("Please enter the statement to be translated:"R_data = result[r_data = result[r_data = result["translateResult"] [0]
        print(r_data[0] ["src"])
        print(r_data[0] ["tgt"])

Copy the code

I’m not going to explain this code too much, but I think you can understand it.

4.1. Implementation effect

5, JS decryption after the complete code upgrade version

== If you just translate some words and sentences, it is better to translate them directly on the website. This upgraded version can translate the article sentences in sections, which is more helpful to improve the ability of English reading comprehension. (especially suitable for students with poor English scores like me) 😂==

import requests
from hashlib import md5
import time
importRandom # request url ="http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule"

appVersion = "5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36"

headers = {
    "Accept": "application/json, text/javascript, */*; Q = 0.01"."Accept-Encoding": "gzip, deflate"."Accept-Language": "zh-CN,zh; Q = 0.9"."Connection": "keep-alive"."Content-Length": "244"."Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"."Cookie": "OUTFOX_SEARCH_USER_ID = - 1506602845 - @10.169.0.82; JSESSIONID=aaaUggpd8kfhja1AIJYpx; OUTFOX_SEARCH_USER_ID_NCOO = 108436537.92676207; ___rl__test__cookies=1597502296408"."Host": "fanyi.youdao.com"."Origin": "http://fanyi.youdao.com"."Referer": "http://fanyi.youdao.com/"."user-agent": appVersion,
    "X-Requested-With": "XMLHttpRequest",}def r(e):
    # bv
    t = md5(appVersion.encode()).hexdigest()

    # lts
    r = str(int(time.time() * 1000))

    # i
    i = r + str(random.randint(0.9))

    return {
        "ts": r,
        "bv": t,
        "salt": i,
        "sign": md5(("fanyideskweb" + e + i + "]BjuETDhU)zqSxf-=B#7m").encode()).hexdigest()
    }


def fanyi(word):

    data = r(word)
    params = {
        "i": word,
        "from": "UTO"."to": "AUTO"."smartresult": "dict"."client": "fanyideskweb"."salt": data["salt"]."sign": data["sign"]."lts": data["ts"]."bv": data["bv"]."doctype": "json"."version": "2.1"."keyfrom": "fanyi.web"."action": "FY_BY_REALTlME",
    }

    response = requests.post(url=url,headers=headers,data=params)
    return response.json()


if __name__ == "__main__"Open the article that needs to be translatedwith open("Article. TXT",mode="r",encoding="utf-8")As f: # get the entire text of the article= f.read()

    result = fanyi(text)
    r_data = result["translateResult"] # Save the translation resultwith open("test.txt",mode="w",encoding="utf-8") as f:
        for data in r_data:
            f.write(data[0] ["tgt"])
            f.write('\n')
            f.write(data[0] ["src"])
            f.write('\n')
            print(data[0] ["tgt"])
            print(data[0] ["src"])

Copy the code

5.1 Implementation effect

The blogger will continue to update, interested partners can == like ==, == attention == = and == = collection == oh, your support is the biggest motivation for my creation!

25 Crawler project treasure Tutorials you Deserve!

The source of this article is available at GitHub github.com/2335119327/… Has included (== more connotation of this blog not crawler, interested partners can see ==), will continue to update, welcome Star.