preface

It has been 3 or 4 years since I last wrote wechat mini program. Recently, I started to write it again. It is quite normal to step on some holes.

(‘ ω · ´) This article will mainly focus on the native development and speech recognition application development process encountered problems.

How to build a speech recognition application

First of all, we abandoned the idea of building a Speech recognition wheel ourselves from the very beginning, and chose to use a third-party API for the functionality (specifically Speech=>Text). After investigation, the voice recognition API of IFLYTEK and the third party plug-in of wechat small program are initially selected.

To be fair, iFLYtek’s API is very powerful, but in the early investigation, found not too good to do… Iflytek’s voice dictation API is based on WebSocket, and the audio format is slightly different from the silk suffix generated by wx.startRecord. That is, to use the relevant API to achieve our functions, after the completion of the recording to first transcode, and then upload, it is a, complex.

At this time, wechat simultaneous interpretation plug-in shining debut!

No transcoding, no Websocket, direct call can achieve speech to text!

Isn’t that attractive… As it turns out, there are a few bumps in the road (though in theory it’s still faster and easier than using the IFlytek API)

Easy steps:

1. Add a third-party plug-in to the mini program management backgroundWechat simultaneous interpretation

2. Modify the app. Json

//app.json{..."plugins": {..."WechatSI": {
            "version": "0.3.1"."provider": "wx069ba97219f66d99"}}}Copy the code

3. Add js to the required page

Example:

//index.js
var plugin = requirePlugin("WechatSI")
let manager = plugin.getRecordRecognitionManager()
Page({
    ...
    onLoad: function() {
        manager.onRecognize = function(res) {
            console.log("current result", res.result)
        }
        manager.onStop = function(res) {
            console.log("record file path", res.tempFilePath)
            console.log("result", res.result)
        }
        manager.onStart = function(res) {
            console.log("Recording identification successfully initiated.", res)
        }
        manager.onError = function(res) {
            console.error("error msg", res.msg)
        }
    },
    startRecord:function(){
        manager.start();
    },
    stopRecord:function(){ manager.stop(); }})Copy the code

Note: recordRecoManager is globally unique; Event binding to the manager is recommended to be implemented on onLoad or other page hooks, but testing in the startRecord method has negative effects.

Well, at this point speech recognition should be implemented on the real phone if nothing else – yes, for unknown reasons, this feature is a puzzle on the applets and has never been successfully identified, so it can only be developed using real machine debugging methods.

If it were just ordinary speech recognition, it would be over by now, but… We also need to allow users to play back their recordings…

Aye? Doesn’t the manager.onStop method above already have a callback res.tempFilePath?

Look at this Warning:

After studying the recording files returned by onStop, we found that nothing was actually heard — and this Warning does not occur every time!! Imagine how confused I was when I couldn’t hear the playback at all on the developer tools and couldn’t recognize the results due to an unexplained bug… In any case, it seemed that I had written the code wrong, so I began to debug the code for a long time, until a chance happened to trigger a Warning…

Although this sentence seems to be on the real machine to be able to hear the playback of the recording, but due to the constraints of the test environment, or additional call wx.startRecord to meet the user to hear the playback of the demand, that is, when the plug-in onStart began to identify, we also call the recording interface, to generate another recording file.

It has been proved by practice that the two can coexist. So you can already hear the recording on Wx. PlayVoice

PS: The official document saysplayVoiceThe method is no longer maintained, but the actual measurements can continue to be used, if more functionality is requiredwx.createInnerAudioContext()Instead.

2. Some problems encountered in native development

Originally, I wanted to try uni-App to do the development of this small program after listening to others’ amway, but then I evaluated the functions to be realized according to our needs. Based on my years of program development experience without data support, The third party framework is the biggest performance bottleneck for our “small and beautiful” program…… Therefore, based on the actual situation, WE chose WXML + WXSS + WXS as a set of original family barrel to do development.

So this section mainly documents some problems related to applets native (:3[

1. WXSS’s imperfections on pseudo-class selectors, such as:hover or :active

WXSS does not support pseudo-class selectors in official documentation. Theoretically, such as :hover,:focus and :active do not work.

However, in practice, it is found that for pseudo-class selectors, wechat applet is not completely unsupported, it should be said that support is not complete.

.floating-panel:hover{
  background-color: #4abcdd;
}
.floating-panel:focus{
  background-color: #7d7d7d;
}
.floating-panel:active{
  background-color: # 343434;
}
Copy the code

Hover does not trigger :hover on the emulator. Moving the cursor after clicking “active” will instead trigger :hover… Others, such as first-child, did not experiment one by one, but the big brothers and sisters in the forum said they could.

Although to be fair, in the use of small programs in wechat, :hover is indeed weak, but in some scenarios, such as button clicking, pseudo-class selector seems to be quite necessary, right?

After careful, patient and careful baidu research, there are two main solutions:

  1. Official Tips: It is recommended to use built-in appletshover-classProperty to implement, discard:active
  2. Folk remedies: Use a data-driven approach, using JS(i.e. add a bidirectional binding to a node's class)To achieve:
<! --Example-->
<view class="choose {{num==1? 'active':''}}" data-num="1" bindtap="itemClick"></view>
Copy the code

2. Direct operations on DOM nodes are not supported

Wechat applets do not support direct manipulation of DOM nodes. If you want to add or subtract page elements, you should also use a data-driven circuitry, i.e Wx :if or wx:for.

3. The <text> tag only supports the <text> tag!

One day, I accidentally put a inside the and the contents wrapped in magically disappeared… My first thought was that I had written some magical CSS that caused the bug, so I reviewed and tested it for half an hour — until I gave up trying to change it back to

. Then I saw this sentence somewhere in the official documentation: “Text component only supports text nesting”.

Yes, not only ,< I >,, but also all other tags except

.

4. The resource path cannot contain Chinese characters, including the file name

Since the interface of the development simulator and my ios test machine is normal, I didn’t notice this problem at the beginning. Until I noticed that there were picture resources on android phones that could not be loaded successfully, I changed some pictures with Chinese names to English and fixed the bug… So the path to the UK is really the blood and tears of the predecessors ah… ╥ man ╥

5. Cloud development

emmmm… Cloud development should also be part of native development.. ? The cloud development of wechat applets is based on NodeJS. Basically, the development tool starts with a few simple examples. Here are a few Tips:

  • You’ll be prompted to install it when you first apply itwx-server-sdkBut it’s not actually installed oncloudfunctionsIn the folder, but in the directory of each cloud functionnpm installSimilarly, each cloud function has its ownpackage.jsonandnode_modules
  • If you want to try a database that has never been developed in the cloud before, don’t forget to create a new Collection in the database first. Each Collection can be viewed as a table in a relational database or as an array in JSON

conclusion

In 2021, the development of micro channel small program is very convenient and fast, can not help but remind me of the micro channel X5 engine bugs were severely lambasted the remarkable years.

Anyway, I hope it was helpful

Disclaimer: because I have not done small program development in the past two years, also did not carefully read the document, if there is an error, it is normal, please friends spray for hope.