Professional search, parents good helper

Long did a picture recognition project, there was a function is the whole problem identification, namely incoming math problem screenshots, can recognize the image content of by OCR technology, but only to identify the text at that time, did not make further processing, now want to practicality is not strong, after all, we need more of is their thinking, Rather than let AI read the question dry (the question dry text, I know, connect up I don’t know how to start to do = =), recently just have time, so try to have a baby friend to do a search artifact.

In view of the good experience of using Youdao Wisdom Cloud in the development of whole question recognition before, I opened its official document again and found the open API of photo question search service, and made a simple batch question search demo with ease and familiarity. The development process will be shared below.

Preparations for calling the API

First of all, you need to create an instance, create an application, bind the application and instance on the personal page of Youdao Wisdom Cloud, and obtain the ID and key of the application. See the article for details about individual registration and application creation

Detailed introduction of the development process

The following describes the specific code development process.

The parameters received by the API are simple:

The field name type meaning mandatory note
q text To recognize the image, Base64 encoding is required True Must be Base64 encoding (do not prefix data:image/ PNG before baes64; base64)
appKey text Application ID True Can be inApplication managementTo view
salt text UUID True uuid
curtime text Current UTC timestamp in seconds true TimeStamp
sign text Signature sha256(app ID+input+salt+curtime+ app key); For the rules for generating input, see the remarks below True Sha256 (App ID+input+salt+curtime+ app key)
signType text Signature type true v2
type text Upload type. Only base64 upload is supported. Set this parameter to a fixed value of 1 True 1
searchType text Search type: IMG is an image search and text is a text search false img

Sign is generated as follows: signType=v2; Sign =sha256(app ID+input+salt+curtime+ app key) Input = the first 10 characters of Q + q length + the last 10 characters of Q (when Q length is greater than 20) or input=q character string (when Q length is less than or equal to 20).

Note that the API has the following requirements for the title image:

The rules describe
transport HTTPS
Request way POST
A character encoding Use utF-8 encoding
The request format The form
The response format JSON
Image format jpg/png/bmp
Image size Under 1 MB
Word length Less than 50 characters

The Demo development:

This demo is developed using python3, including maindow. Py, questionclass. py, OcrQuestion.

  1. Interface part:

    UI part is relatively simple, the main function is to select the picture to be subject, select the correction result storage path. The layout code is as follows:

    root=tk.Tk()
root.title(" youdao ocr question test")
frm = tk.Frame(root)
frm.grid(padx='50', pady='50')

# Select topic and save result button
btn_get_file = tk.Button(frm, text='Select subject picture', command=get_files)
btn_get_file.grid(row=0, column=0, ipadx='3', ipady='3', padx='10', pady='20')
text1 = tk.Text(frm, width='40', height='10')
text1.grid(row=0, column=1)
btn_get_result_path=tk.Button(frm,text='Select search result path',command=set_result_path)
btn_get_result_path.grid(row=1,column=0)
text2=tk.Text(frm,width='40', height='2')
text2.grid(row=1,column=1)


# Search button
btn_sure=tk.Button(frm,text="Search the topic",command=search_question_files)
btn_sure.grid(row=4,column=1)

root.mainloop()
Copy the code

    Where, start the binding event search_question_files() of button btn_sure to search questions according to the question photos, and open the result storage path after completion:

    def search_question_files() :
    question.start_ocr()
    os.system('start '+question.result_path)
Copy the code
  2. QuestionClass.py

    Here, mainly with the logic of UI, call the search method.

    Start by defining a class Question:

    class Question() :
    def __init__(self,file_paths,result_path) :	
        self.file_paths=file_paths		# Title photo storage path
        self.result_path=result_path	# result path
Copy the code

    The start_ocr() method calls the connect() method to search the questions in turn and save the results.

    def start_ocr(self) :
    for file_path in self.file_paths:
        result=connect(file_path)
        print(file_path)
       self.save_result_format(file_path,result)
Copy the code

    The result obtained from ocrquestion.py’s connect method is json format, save_result_format() method parses the interface obtained from the interface, reformats, and saves the result to HTML:

        def save_result_format(self,file_path,result) :
        result_file_name=os.path.basename(file_path).split('. ') [0] +'_result.html'
        f=open(self.result_path+'/'+result_file_name,'w',encoding='utf-8')
        result_json= json.loads(result)
        if result_json['errorCode'] = ='0':
            data=result_json['data']
            questions=data["questions"]
            text=data["text"]
            f.write("

    "    +text)
            i=0
            for answers in questions:
                i=i+1
                subject="Subject:"+answers["subject"] +"<br>"
                answer="Answer:" +answers["answer"] +"<br>"
                analysis="Analysis:"+answers["analysis"] +"<br>"
                knowledge="Knowledge point:"+answers["knowledge"] +"<br>"
                print(subject+answer+analysis+knowledge)
                result_each="< H3 > Results"+str(i)+"<br></h3>"
                result_each=result_each+subject+answer+analysis+knowledge+"

    ================= this is a delimiter ============

    "    
                f.write(result_each)
        else:
            f.write("result error code:"+result_json['errorCode'])
Copy the code
  3. OcrQuestion.py

    Ocrquestion.py encapsulates the methods that request the OCR search API, the main of which is connect() :

    def connect(pic_path) :
    f = open(pic_path, 'rb')  Open the image file in binary mode
    q = base64.b64encode(f.read()).decode('utf-8')  Read the content of the file and convert it to Base64 encoding
    f.close()

    data = {}
    data['q'] = q
    data['signType'] = 'v2'
    curtime = str(int(time.time()))
    data['curtime'] = curtime
    salt = str(uuid.uuid1())
    signStr = APP_KEY + truncate(q) + salt + curtime + APP_SECRET
    sign = encrypt(signStr)
    data['appKey'] = APP_KEY
    data['salt'] = salt
    data['sign'] = sign

    response = do_request(data)
    result=response.content.decode('utf-8')
    print(result)
    return result
Copy the code
    Example API response results
    {
    "data": {"questions":[
            {
                "score":0.9875."answer":"D"."subject":"History"."id":"a9db8f1252778836c99204e5cf9d7738"."analysis":""."type":""."content":"Scholars believe that the May 4th Movement marked the great awakening of Chinese national consciousness, as shown in < BR />A. Modern national democratic thought began to spread B. Opened the curtain of the struggle against imperialism and feudalism. Opened the ideological enlightenment of modern China. Broad mobilization and participation of all sectors of the population."."knowledge":""
            },
            {
                "score":0.9875."answer":During the May 4th Movement, students, workers, and businessmen were all involved in the mobilization and participation of people from all walks of life. < BR > The idea of national democracy in China began to spread around the Hundred Days' Reform. < BR > The May 4th Movement was a patriotic movement against imperialism and feudalism, but it was not the beginning of the struggle against imperialism and feudalism. < BR > In modern Times, China spread the idea of democracy during the New Culture Movement and carried out ideological enlightenment. 

     D."    ."subject":"History"."id":"2f8b9c814f064131cc7b2bead54f15da"."analysis":The May 4th Movement. The Significance of the May 4th Movement. Test students' ability to memorize and use basic knowledge.."type":"Multiple choice"."content":"Scholars believe that the May 4th Movement marked the great awakening of Chinese national consciousness, which is manifested in () question_option< p style = "max-width: clear; clear: both; min-height: 1em; 

    D. The widespread mobilization and participation of people from all walks of life 
        "    ."knowledge":"/ The history of human political civilization/The trend of anti-aggression and democracy in modern China/the May Fourth Movement"
            },
            {
                "score":0.9875."answer":
    D
     
     
     
    ."subject":"History"."id":"48a28969d882f20b04ffec15c8f1b94c"."analysis":< P > During the May 4th Movement of 1898, China's national democratic ideology began to spread. The May 4th Movement was a patriotic movement against imperialism and feudalism, but it was not the beginning of the struggle against imperialism and feudalism. In modern China, democracy was spread and enlightenment was carried out during the New Culture Movement. During the May 4th Movement, students, workers and businessmen were all involved. 
     
     
    "    ."type":"Multiple choice"."content":"
     Scholars believe that the May 4th Movement marked the great awakening of Chinese national consciousness, as shown in 
     
         question_option< p style = "max-width: clear; clear: both; min-height: 1em; 

    D. The widespread mobilization and participation of people from all walks of life 
        "    ."knowledge":"/ The history of political civilization of mankind/The trend of anti-aggression and Democracy in modern China/The First National Congress of the Communist Party of China"}]."text":"Scholars believe that the May 4th Movement marked the great awakening of China's national consciousness, manifested in (A) the spread of modern national democratic thought, (B) the prelude to the anti-imperialist and anti-feudal struggle, (C) the ideological enlightenment of modern China. Broad mobilization and participation of all sectors of the population."
    },
    "errorCode":"0"
}
Copy the code

    The response result is output in JSON format, including fields as shown in the following table:

field meaning
errorCode Identify the result error code, it must exist. For details, seeError code list
data data
-text Picture title OCR results
-questions Related topics
– id The answer
– the content The subject content
– answer The answer
– analysis parsing
– knowledge knowledge

Results show

Demo Operation Demonstration

Check out the results:

Math search results display:

History questions search results

conclusion

The overall search API of Youdao Wisdom Cloud has clear documentation, a wide range of topics and can automatically judge disciplines. The search results can be described as “drawing inferring from one’s own experience”, and several possible similar topics will be returned, which is of great reference value and worth recommending! Math solutions return images, formulas, and so on, which can work well in Web projects.

Project address: github.com/LemonQH/Ocr…