Abstract: Developers can develop an elastic and highly available back-end system by using functional workflow to build applications in a serverless manner without configuring and managing servers. Managed functions can operate in a manner of millisecond elastic scaling, free operation and maintenance, and high reliability, which greatly improves the efficiency of development and operation and maintenance and reduces the operation cost.
FunctionGraph (FGS) is an event-driven, function-driven computing service that manages functions in a scalable, operation-free, and highly reliable manner. With functional workflows, developers don’t need to configure and manage servers. Instead, they can focus on business logic, write functional code, build applications in a serverless manner, and develop an elastic, highly available back-end system that can be charged by the resources actually consumed. Greatly improve the development and operation efficiency, reduce the operation cost.
Compared with traditional architecture, the serverless architecture built by functional workflow has the following advantages:
1. Do not need to pay attention to any server, just focus on the core business logic, improve development and operation efficiency;
2. Function operation is flexible and scalable with the volume of business, pay as you need, and charge only after execution, which can reduce a lot of cost for the scene with obvious peaks and troughs of traffic;
3. Function workflow and other cloud services can be connected through simple configuration, or even cloud services and cloud services;
In order to further let you feel the advantages of functional workflow, we will introduce how to quickly build a serverless sensitive word filtering system through functional workflow. In this paper, we mainly focus on the back-end system, the front-end performance of many forms, you can build their own. The system identifies whether the text you upload contains sensitive information (such as pornography or politics) and filters those words.
Imagine if we were to develop this application in a traditional way.
Even now is based on the cloud platform, we still need to purchase a cloud server, pay close attention to its specifications, mirror, network and so on each index selection and ops, and then in the development process may also need to consider the integration with other cloud services use problem, make the code coupling in non-business code, and the server resources such as also is not as required, Especially for the scene where the peaks and troughs of traffic are very obvious, there will be a lot of extra costs.
Now we can build this system quickly with a functional workflow service, completely server-free, flexible and on-demand, as shown in figure:
Create a function to invoke the text detection interface provided by Huawei Cloud content detection service in the function to realize text sensitive word detection, configure an APIG trigger for the function, and provide the API of sensitive word filtering externally, so as to build a complete serverless system of sensitive word filtering. The client calls the API, which automatically triggers the function execution, and the developer writes the function to implement the logic of how to process the text once it is received (calling the content detection service service), and finally returns the result to the client. At this point, we have built a complete serverless sensitive word filtering system.
Now, we’ll show you how to build this serverless system end-to-end.
1. Preparation
Log in to huawei cloud content detection service and apply for enabling text content detection. After the application is successful, the text detection interface provided by the content detection service can be invoked.
2. Create functions
Enter the function workflow service page, create a function, implement text detection interface call and sensitive word filtering, code is as follows (Python) :
# -*- coding:utf-8 -*-
import json
import base64
import urllib
import urllib2
import ssl
import sys
reload(sys)
sys.setdefaultencoding(‘utf-8’)
def do_filter(msg,str_list):
result = ”
try:
if len(str_list) <=0:
return msg
for str in str_list:
str_tmp = msg.replace(str,”)
msg = str_tmp
result = msg
except:
print(“_do_filter catch an exception!” )
return result
def filter(context, msg):
result = ”
try:
ssl._create_default_https_context = ssl._create_unverified_context
token = context.getToken();
headers = {‘Content-Type’:’application/json; charset=utf8′,’X-Auth-Token’:token}
Url = “https://ais.cn-north-1.myhwclouds.com/v1.0/moderation/text”
values = {}
values[‘categories’] = [‘porn’,’ad’,’politics’,’abuse’,’contraband’]
#msg = base64.b64encode(msg)
item = {‘type’:’content’,’text’:msg}
values[‘items’] = [item]
data = json.dumps(values)
print(“data: %s”%data)
request = urllib2.Request(url,data,headers)
rsp = urllib2.urlopen(request)
http_rsp = rsp.read()
print(“http response: %s” %http_rsp)
json_rsp = json.loads(http_rsp)
result = json_rsp[‘result’]
suggestion = result[‘suggestion’]
if suggestion == ‘pass’:
print(“input msg have passed the checking!” )
result = msg
else:
detail = result[‘detail’]
if detail.has_key(‘porn’):
list_porn = detail[‘porn’]
msg = do_filter(msg,list_porn)
if detail.has_key(‘ad’):
list_ad = detail[‘ad’]
msg = do_filter(msg,list_ad)
if detail.has_key(‘politics’):
list_politics = detail[‘politics’]
msg = do_filter(msg,list_politics)
if detail.has_key(‘abuse’):
list_abuse = detail[‘abuse’]
msg = do_filter(msg,list_abuse)
if detail.has_key(‘contraband’):
list_contraband = detail[‘contraband’]
msg = do_filter(msg,list_contraband)
result = msg
except Exception, e:
print e
print(“filter catch an exception!” )
return result
def handler (event, context):
print(“message filter begin!” )
result = “”
response = {}
http_method = event.get(‘httpMethod’)
if http_method == ‘OPTIONS’:
response = {
‘statusCode’: 200,
‘isBase64Encoded’: True,
‘headers’: {
“Content-Type”: “application/json; charset=utf-8”,
“Access-Control-Allow-Origin”: “*”,
“Access-Control-Allow-Headers”: “Content-Type,Accept”,
“Access-Control-Allow-Methods”: “GET,POST,PUT,DELETE”
},
‘body’: base64.b64encode(‘{“result”:’+ ‘”‘ + result +'”}’),
}
return response
body = event.get(‘body’)
body_decode = base64.b64decode(body)
json_object = json.loads(body_decode)
msg = json_object[‘msg’]
print(‘msg : %s’%msg)
try:
result = filter(context, msg)
response = {
‘statusCode’: 200,
‘isBase64Encoded’: True,
‘headers’: {
“Content-Type”: “application/json; charset=utf-8”,
“Access-Control-Allow-Origin”: “*”,
“Access-Control-Allow-Headers”: “Content-Type,Accept”,
“Access-Control-Allow-Methods”: “GET,POST,PUT,DELETE”
},
‘body’: base64.b64encode(‘{“result”:’+ ‘”‘ + result +'”}’),
}
except:
print(“function catch an exception!” )
return response
After the function is created, configure a delegate with IAM access permission for it, because the ak and SK of the user in this function code need to have IAM access permission.
3. Create the APIG trigger
Configure an APIG trigger for the function, resulting in an HTTP (S) API that calls the function for external calls.
4. Test
Use tools such as Postman to send a POST request to the interface of the APIG trigger created in the previous step. The body is {” MSG “:” filter detected text “}, and check the returned information.
Like sending {” MSG “: “just fuck…” }, return body {“result”: “just…” }
Since then, we have completely implemented a serverless sensitive word filtering back-end system.
Huawei Cloud content check: t.cn/Re8RHHV
Huawei Cloud function workflow: t.cn/Re8R317