Recently, Xiaobian encountered a survival problem. My girlfriend asked me to translate English cosmetics labels for her. It is called: “program ape every day English development, English must be very good, to help me translate the makeup ingredients”, “come, help me to have a look at this mask suggested to apply a few minutes”… It seems that it is not enough to spend a huge sum of money on cosmetics, but also need to learn various English introductions.

Silently put away a pile of college entrance examination 429 points of the four level certificate, I opened the IDE… I am going to develop a demo that can translate pictures in batches and translate all kinds of cosmetics at home. As smart as I am, I would not start from the training model. When I opened the friendly AI interface page of Youdao Wisdom Cloud, there was a picture translation service. After experiencing it, it was really good, so I decided to use it.

Results show

The Demo is here. Let’s see what it looks like:

The identification process is as follows:

Look at the effect one by one! Make up for ever though not translated into Mei Ke fi, ha ha ha, but the key words long-term moisturizing, fixed spray are translated ~~ rod

This is more unknown, Korean, English can be translated

Sakura water also performed well

Add an image recognition that opens more like a box, and the effect is good, not affected by the text tilt on the picture, etc. :

Preparation of the call API – generate the application ID and key needed for the call

According to the interface agreement of Youdao Wisdom Cloud, you need to generate the application ID and key required for calling on youdao Wisdom Cloud’s personal page first, so as to be used as your calling mark and charging reference.

The specific steps are as follows: create an instance, create an application, bind the application and instance on the personal page of Youdao Wisdom Cloud, and obtain the ID and key of the application used to invoke the interface. For details about the process of individual registration and application creation, see the development process of a batch file translation shared in the article

Introduction to development Process

1. API interface introduction

First introduce the core part of the project, the call interface of Youdao Wisdom Cloud image translation service

API HTTPS address: openapi.youdao.com/ocrtransapi

Interface call mode: POST

Request format: form

Corresponding format: JSON

Interface call parameter

Calling the API requires sending the following fields to the interface to access the service.

Parameter Type Meaning Remarks TypeText File upload type True Currently Base64 is supported. Set the value of this field to 1fromText Source language True Refer to the supported language below (auto can be set) toText Target language True Refer to the supported language below (auto can be set)appKeytext Application IDTrue Can be used to manage applications See salttextUUIDTrue1995882C5064805BC30A39829B779D7Bsigntext signature Truemd5 (Id + q + salt + application key) exttext translation results audio formats, Mp3falsemp3qtext Image to be identified This parameter is mandatory when trueType is 1 and the Base64 encoding of the image docTypetext server response type. Currently, only jSONFalseJsonRenderText is supported. 0: No False0nulliserrortext Indicates whether to return an error if the OCR does not detect a text. False: No. True: Yes, which is a string by default

The signature generation method is as follows:

1. Concatenate the application ID appKey, Base64 encoding Q, UUID salt and application key in the request parameters in the sequence of application ID+ Q +salt+ application key to obtain the string STR.

2. Perform MD5 on string STR to get 32-bit uppercase sign (see Java md5 generation example, click Java example).

The output

The returned result is in JSON format, described as follows:

Field name field description orientation orientation of the image the language lanFromocr identifies in the image textAngle the tilt Angle of the image errorCode errorCode lanTo target language resRegions specific content of the image translation -boundingBox area Range, four values: the x value in the upper left corner, the Y value in the upper left corner, the width of the region, and the height of the region -linesCount number of lines (used for front end typesetting) -lineheight lineheight -context text of this area -linespace line spacing -tranContent translation result

2. Detailed development

This demo is developed using PYTHon3 and includes maindow. Py, transclass.py, and pictranslate. Maindow. Py implements the interface, using python’s tkinter library to select image files and store the results. Transclass. py implements the logic of image reading and processing, and finally calls the image translation API through methods in pictranslate.

1. Interface

Main elements:

root=tk.Tk() root.title("netease youdao translation test") frm = tk.Frame(root) frm.grid(padx='50', Pady ='50') btn_get_file = tk.Button(FRM, text=' select image ', command=get_files) btn_get_file.grid(row=0, column=0, ipadx='3', ipady='3', padx='10', pady='20') text1 = tk.Text(frm, width='40', height='10') text1.grid(row=0, Column =1) btn_get_result_path=tk.Button(FRM,text=' select result_path ',command=set_result_path) btn_get_result_path.grid(row=1,column=0) text2=tk.Text(frm,width='40', Grid (row=1,column=1) btn_sure=tk.Button(FRM,text=" translation ",command=translate_files) btn_sure.grid(row=2,column=1) root.mainloop()Copy the code

Method for obtaining image files to be translated (only. JPG files are supported here) :

def get_files(): files = filedialog.askopenfilenames(filetypes=[('text files', '.jpg')]) translate.file_paths=files if files: For file in files: text1.insert(tk.end, file + '\n') text1.update() else: print(' You did not select any file ')Copy the code

Obtain the result storage path:

def set_result_path():
    result_path=filedialog.askdirectory()
    translate.result_root_path=result_path
    text2.insert(tk.END,result_path)
Copy the code

The translate_files() method in this file finally calls the translate_files() method of translate:

Def translate_files(): if translate.file_paths: translate.translate_files() tk.messagebox.showinfo(" "," "") else: Tk.messagebox.showinfo (" prompt "," no file ")Copy the code

2. Batch image processing

Transclass. py implements image reading and processing logic. Translate class definitions are as follows:

class Translate(): def __init__(self,name,file_paths,result_root_path,trans_type): Self.name =name self.file_paths=file_paths # Path of the file to be translated self.result_root_path=result_root_path # Path where the result is stored self.trans_type=trans_type def translate_files(self): for file_path in self.file_paths: # for bulk picture processing file_name = OS one by one. The path. The basename (file_path) print (" = = = = = = = = = = = '+ file_path +' = = = = = = = = = = = ') Trans_reult =self.translate_use_netease(file_path) # Call the interface for a single image Resul_file =open(self.result_root_path+'/result_'+file_name.split('.')[0]+'.txt','w').write(trans_reult) # return result to write def Translate_use_netease (self,file_content): # Call youdao interface and return result= connect(file_content) return resultCopy the code

3. Youdao API calls

Pictranslate. Py encapsulates some methods to call The YOUdao Wisdom Cloud API, among which the most core is connect() method, which splices the required parameters according to the interface requirements, initiates the request and returns the result.

def connect(file_content,fromLan,toLan): F = open(file_content, 'rb') # open the image file in binary mode q = base64.b64encode(f.read()).decode(' utF-8 ') # Converted into base64 encoding f. lose () data = {} # data [' from '] = 'the source language' # data [' to '] = 'target language data [' from'] = 'auto' data [' to '] = 'auto' data['type'] = '1' data['q'] = q salt = str(uuid.uuid1()) signStr = APP_KEY + q + salt + APP_SECRET sign = encrypt(signStr) data['appKey'] = APP_KEY data['salt'] = salt data['sign'] = sign response = do_request(data) result=json.loads(str(response.content, encoding="utf-8")) print(result) translateResults=result['resRegions'] print(translateResults) pictransresult="" for i in translateResults: pictransresult=pictransresult+i['tranContent']+"\n" return pictransresultCopy the code

conclusion

Is a pleasant development experience, and is one of the few survival experience success: P, unexpectedly with the power of open platform, image recognition, natural language processing has become so easy, as long as can initiate requests correctly, you can get a good translation as a result, the remaining time is used to show off with his girlfriend, this feeling great!