For reprint please indicate the CSDN post address: blog.csdn.net/ls0609/arti…
Online listening demo:blog.csdn.net/ls0609/arti… Voice demo:blog.csdn.net/ls0609/arti charge to an account…
Android desktop suspension window implementation is relatively simple, this paper with a speech recognition, semantic understanding of the demo to demonstrate how to achieve Android suspension window.
1. Suspension window effect
When the desktop is on standby, the suspension window sticks to the side
When you drag it away from the edge of the screen, the icon becomes larger. When you release it, it automatically runs to the edge of the screen and absorbs the side close to the left and right edges of the screen
Click the hover icon to start recording
After finishing, you can click the left button to upload the recording to the server and wait for the result of processing
After the server returns a result, the application page is displayed. In this example, the page is displayed for online reading
2. FloatViewIdle and FloatViewIdleService
1.FloatViewIdle Defines a FloatViewIdle class. The following is a singleton of this class
public static synchronized FloatViewIdle getInstance(Context context)
{
if(floatViewManager == null)
{
mContext = context.getApplicationContext();;
winManager = (WindowManager)
mContext.getSystemService(Context.WINDOW_SERVICE);
displayWidth = winManager.getDefaultDisplay().getWidth();
displayHeight = winManager.getDefaultDisplay().getHeight();
floatViewManager = new FloatViewIdle();
}
return floatViewManager;
}Copy the codeUsing the AddView method of winManager, add a custom FloatView to the screen to display the floatView in any interface. Then hide the FloatView by masking the non-standby interface so that only the standby window is displayed.
Define two custom Views, namely FloatIconView and FloatRecordView. The former is the small icon icon seen in standby mode, and the latter is the interface of recording displayed after clicking the icon icon.
So how do I define FloatIconView
class FloatIconView extends LinearLayout{
private int mWidth;
private int mHeight;
private int preX;
private int preY;
private int x;
private int y;
public boolean isMove;
public boolean isMoveToEdge;
private FloatViewIdle manager;
public ImageView imgv_icon_left;
public ImageView imgv_icon_center;
public ImageView imgv_icon_right;
public int mWidthSide;
public FloatIconView(Context context) {
super(context);
View view = LayoutInflater.from(mContext).
inflate(R.layout.layout_floatview_icon, this);
LinearLayout layout_content =
(LinearLayout) view.findViewById(R.id.layout_content);
imgv_icon_left = (ImageView) view.findViewById(R.id.imgv_icon_left);
imgv_icon_center = (ImageView) view.findViewById(R.id.imgv_icon_center);
imgv_icon_right = (ImageView) view.findViewById(R.id.imgv_icon_right);
imgv_icon_left.setVisibility(View.GONE);
imgv_icon_center.setVisibility(View.GONE);
mWidth = layout_content.getWidth();
mHeight = layout_content.getHeight();
if((mWidth == 0)||(mHeight == 0))
{
int temp = DensityUtil.dip2px(mContext, icon_width);
mHeight = temp;
icon_width_side_temp = DensityUtil.dip2px(mContext, icon_width_side);
mWidth = icon_width_side_temp;
}
manager = FloatViewIdle.getInstance(mContext);
if(params! =null)
{
params.x = displayWidth - icon_width_side_temp;
params.y = displayHeight/2; }}public int getFloatViewWidth()
{
return mWidth;
}
public int getFloatViewHeight()
{
return mHeight;
}
@Override
public boolean onTouchEvent(MotionEvent event)
{
switch(event.getAction())
{
case MotionEvent.ACTION_DOWN:
preX = (int)event.getRawX();
preY = (int)event.getRawY();
isMove = false;
if(params.width == icon_width_side_temp)
handler.sendMessage(handler.obtainMessage(
MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED, 3.0));
break;
case MotionEvent.ACTION_UP:
if(isMoveToEdge == true)
{
if(params.width == icon_width_side_temp)
handler.sendMessage(handler.obtainMessage(
MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED, 3.0));
handler.sendMessage(handler.obtainMessage(
MSG_FLOAT_VIEW_MOVE_TO_EDGE,this));
}
break;
case MotionEvent.ACTION_MOVE:
x = (int)event.getRawX();
y = (int)event.getRawY();
if(Math.abs(x-preX)>1||Math.abs(y-preY)>1)
{
isMoveToEdge = true;
}
if(Math.abs(x-preX)>5||Math.abs(y-preY)>5)
isMove = true;
if(params.width == icon_width_side_temp)
handler.sendMessage(handler.obtainMessage(
MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED, 3.0));
manager.move(this, x-preX, y-preY);
preX = x;
preY = y;
break;
}
return super.onTouchEvent(event); }}Copy the codeGenerate a FloatIconView through the layout file. In the onTouchEvent function, when pressed, send a message to update the floating view. When the up event is raised, the floating View will be updated first, and then the animation attached to the side will be displayed. When moving, the view position will be updated according to the judgment that each displacement is at least 5 pixels. In this way, continuous moving and updating will form a continuous picture.
In the same way, the other FloatRecordView FloatRecordView is used to create a FloatRecordView. If you are interested, you can download the demo and run it yourself.
Define a handler in FloatIconView that receives messages to animate the hover window update position and adsorption
private void initHandler(){
handler = new Handler(){
@Override
public void handleMessage(Message msg)
{
switch (msg.what)
{
case MSG_REFRESH_VOLUME:
if(floatRecordView ! =null)
floatRecordView.updateVolume((int)msg.arg1);
break;
case MSG_FLOAT_VIEW_MOVE_TO_EDGE:
// Update the hover window position animation
moveAnimation((View)msg.obj);
break;
case MSG_REMOVE_FLOAT_VIEW:
if(msg.arg1 == 1)
{// A floatView is floatIconView
if(floatIconView ! =null)
{// Remove a floatView first
winManager.removeView(floatIconView);
floatIconView = null;
floatRecordView = getFloatRecordView();
if(floatRecordView ! =null)
{
if(floatRecordView.getParent() == null)
{// Add a new floatView
winManager.addView(floatRecordView, params);
floatViewType = FLOAT_RECORD_VIEW_TYPE;
}
if(mHandler ! =null)
{
mHandler.sendMessage(mHandler.obtainMessage(
MessageConst.CLIENT_ACTION_START_CAPTURE));
IS_RECORD_FROM_FLOAT_VIEW_IDLE = true; }}}}else
{// The existing floatView is floatRecordView, which is the recording floatView
if(floatRecordView ! =null)
{// Remove a floatView first
winManager.removeView(floatRecordView);
floatRecordView = null;
}
floatIconView = getFloatIconView();
if(floatIconView ! =null)
{
if(floatIconView.getParent() == null) {/ add a new floatView winManager.addView(floatIconView,params);
floatViewType = FLOAT_ICON_VIEW_TYPE;
setViewOnClickListener(floatIconView);
}
// There may be a need for adsorption animationmoveAnimation(floatIconView); }}break;
case MSG_UPDATE_VIEW_SENDING_TO_SERVER:
if(floatRecordView ! =null)
{
floatRecordView.updateSendingToServerView();
floatRecordView.setTitle("Trying to identify.");
}
break;
case MSG_UPDATE_ROTATE_VIEW:
if(floatRecordView ! =null)
{
floatRecordView.rotateview.startRotate();
}
break;
case MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED:
// 1,2 is to adsorb to the left or right, 3 is to drag to the middle to display the enlarged suspension window icon
if(msg.arg1 == 1)
changeFloatIconToSide(false);
else if(msg.arg1 == 2)
changeFloatIconToSide(true);
else if(msg.arg1 == 3)
changeFloatIconToNormal();
break;
case MSG_UPDATE_FLOAT_VIEW_ON_SIDE:
if(msg.arg1 == 1)
updateFloatIconOnSide(true);
else if(msg.arg1 == 2)
updateFloatIconOnSide(false);
break;
case MSG_START_ACTIVITY:
hide();
Intent intent = new Intent(mContext,MusicActivity.class);
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
intent.putExtra(START_FROM_FLOAT_VIEW, true);
IS_START_FROM_FLOAT_VIEW_IDLE = true;
mContext.startActivity(intent);
break; }}}; }Copy the codeSo, how to click the suspension button on the edge of the screen to switch to the suspension window for recording?
public void show()
{
isHide = false;
floatIconView = getFloatIconView();
if(floatIconView ! =null)
{
if(floatIconView.getParent() == null)
{
winManager.addView(floatIconView, params);
floatViewType = FLOAT_ICON_VIEW_TYPE;
}
if(floatRecordView ! =null)
{
handler.sendMessage(handler.obtainMessage(
MSG_REMOVE_FLOAT_VIEW, 2.0));
}
floatIconView.setOnClickListener(new OnClickListener(){
@Override
public void onClick(View v) {
if(floatIconView.isMove || floatIconView.isMoveToEdge)
{
floatIconView.isMove = false;
return;
}
winManager.removeView(floatIconView);
floatIconView = null;
floatRecordView = getFloatRecordView();
if(floatRecordView ! =null)
{
if(floatRecordView.getParent() == null)
{
winManager.addView(floatRecordView, params);
floatViewType = FLOAT_RECORD_VIEW_TYPE;
}
if(mHandler ! =null)
{
mHandler.sendMessage(mHandler.obtainMessage(
MessageConst.CLIENT_ACTION_START_CAPTURE));
IS_RECORD_FROM_FLOAT_VIEW_IDLE = true; }}}}); }}Copy the codeIn the show function, set the floatIconView click event, remove the small suspension adsorption button, add the recording window view and start recording.
2.FloatViewIdleService
Why define this service? This service periodically scans the standby desktop to display FloatView if it is in a standby desktop and hides it otherwise.
public class FloatViewIdleService extends Service {
private static Handler mHandler;
private FloatViewIdle floatViewIdle;
private final static int REFRESH_FLOAT_VIEW = 1;
private boolean is_vertical = true;
@Override
public void onCreate() {
super.onCreate();
initHandler();
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
mHandler.sendMessageDelayed(mHandler.obtainMessage(REFRESH_FLOAT_VIEW), 500);
FloatViewIdle.IS_START_FROM_FLOAT_VIEW_IDLE = false;
is_vertical = true;
return START_STICKY;
}
protected void initHandler() {
mHandler = new Handler() {
@Override
public void handleMessage(Message msg) {
switch (msg.what) {
case REFRESH_FLOAT_VIEW://1s Sends a floatView update message
updateFloatView();
mHandler.sendMessageDelayed(
mHandler.obtainMessage(REFRESH_FLOAT_VIEW), 1000);
break; }}}; }private void updateFloatView()
{
boolean isOnIdle = isHome();// Check whether the standby screen is displayed
floatViewIdle = FloatViewIdle.getInstance(FloatViewIdleService.this);
if(isOnIdle)
{ // In standby mode, FloatView is displayed
if(floatViewIdle.getFloatViewType() == 0)
{
floatViewIdle.show();
}
else if(floatViewIdle.getFloatViewType() ==
floatViewIdle.FLOAT_ICON_VIEW_TYPE||
floatViewIdle.getFloatViewType() ==
floatViewIdle.FLOAT_RECORD_VIEW_TYPE)
{
if(this.getResources().getConfiguration().orientation ==
Configuration.ORIENTATION_LANDSCAPE)
{
if(is_vertical == true)
{
floatViewIdle.swapWidthAndHeight();
is_vertical = false; }}else if(this.getResources().getConfiguration().orientation ==
Configuration.ORIENTATION_PORTRAIT)
{
if(is_vertical == false)
{
floatViewIdle.swapWidthAndHeight();
is_vertical = true; }}}}else
{// Otherwise hide floatViewfloatViewIdle.hide(); }}private boolean isHome()
{
ActivityManager mActivityManager = (ActivityManager)
getSystemService(Context.ACTIVITY_SERVICE);
List<RunningTaskInfo> rti = mActivityManager.getRunningTasks(1);
try{
if(rti.size() == 0)
{
return true;
}else
{
if(rti.get(0).topActivity.getPackageName().
equals("com.olami.floatviewdemo"))
return false;
else
return getHomes().contains(rti.get(0).topActivity.getPackageName()); }}catch(Exception e)
{
return true; }}private List<String> getHomes()
{
List<String> names = new ArrayList<String>();
PackageManager packageManager = this.getPackageManager();
Intent intent = new Intent(Intent.ACTION_MAIN);
intent.addCategory(Intent.CATEGORY_HOME);
List<ResolveInfo> resolveInfo = packageManager.queryIntentActivities(intent,
PackageManager.MATCH_DEFAULT_ONLY);
for (ResolveInfo ri : resolveInfo) {
names.add(ri.activityInfo.packageName);
}
return names;
}
@Override
public void onDestroy() {
super.onDestroy();
if(floatViewIdle ! =null)
floatViewIdle.setFloatViewType(0);
}
@Override
public IBinder onBind(Intent intent) {
return null; }}Copy the code3. Enable speech recognition
In another VoiceSdkService (another service that handles the recording service), when a hover window button click event message is received, the recording service is started and the result returned by the server is received in the onResult callback after recording.
This example uses OlAMI speech recognition, semantic understanding engine, OlAMI supports powerful user-defined semantics, can better solve semantic understanding. For example, I want to listen to the romance of The Three Kingdoms, I want to listen to the romance of The Three Kingdoms, there are many similar statements, Olmai can solve this kind of semantic understanding for you, olami speech recognition engine is relatively easy to use, just need to simple initialization, and then set up a callback listener, The json string returned by the server is handled during the callback, but the semantics are user-defined.
public void init()
{
initHandler();
mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);
TelephonyManager telephonyManager=(TelephonyManager) this.getSystemService(
(this.getBaseContext().TELEPHONY_SERVICE);
String imei=telephonyManager.getDeviceId();
mOlamiVoiceRecognizer.init(imei);// Set the identity, can be null
mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);// Set the recognition result callback listener
mOlamiVoiceRecognizer.setLocalization(
OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);// Set the supported voice type. Simplified Chinese is preferred
mOlamiVoiceRecognizer.setAuthorization("51a4bb56ba954655a4fc834bfdc46af1"."asr"."68bff251789b426896e70e888f919a6d"."nli");
// Register the Appkey, which is generated after the application is registered on olami website
// Register API, please directly fill in "ASR" to identify the voice recognition type
// Register secret, which is generated after registering the application on olami's official website
// To register seQ, please fill in "NLI"
mOlamiVoiceRecognizer.setVADTailTimeout(2000);// end time of the last voice when recording. 2000ms is recommended
// Set latitude and longitude information, do not want to upload location information, can be 0
mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498.121.34882432933009); Copy the codeDefined in VoiceSdkService OlamiVoiceRecognizerListener callback for dealing with the recording
OnError (int errCode) OnEndOfSpeech ()// End of recording onBeginningOfSpeech()// Start of recording onResult(String result, OnUpdateVolume (int Volume)// Volume at recording levels. The value ranges from 1 to 12
This article uses the example of online listening. When receiving the message returned by the server, enter the following functions: In the following functions, the user needs to extract the semantic understanding fields for processing by parsing the JSON string returned by the server
private void processServiceMessage(String message)
{
String input = null;
String serverMessage = null;
try{
JSONObject jsonObject = new JSONObject(message);
JSONArray jArrayNli = jsonObject.optJSONObject("data").optJSONArray("nli");
JSONObject jObj = jArrayNli.optJSONObject(0);
JSONArray jArraySemantic = null;
if(message.contains("semantic"))
jArraySemantic = jObj.getJSONArray("semantic");
else{
input = jsonObject.optJSONObject("data").optJSONObject("asr").
optString("result");
sendMessageToActivity(MessageConst.
CLIENT_ACTION_UPDATA_INPUT_TEXT, 0.0.null, input);
serverMessage = jObj.optJSONObject("desc_obj").opt("result").toString();
sendMessageToActivity(MessageConst.
CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 0.0.null, serverMessage);
return;
}
JSONObject jObjSemantic;
JSONArray jArraySlots;
JSONArray jArrayModifier;
String type = null;
String songName = null;
String singer = null;
if(jObj ! = null) {
type = jObj.optString("type");
if("musiccontrol".equals(type))
{
jObjSemantic = jArraySemantic.optJSONObject(0);
input = jObjSemantic.optString("input");
jArraySlots = jObjSemantic.optJSONArray("slots");
jArrayModifier = jObjSemantic.optJSONArray("modifier");
String modifier = (String)jArrayModifier.opt(0);
if((jArrayModifier ! = null) && ("play".equals(modifier)))
{
if(jArraySlots ! = null)
for(int i=0,k=jArraySlots.length(); i<k; i++)
{
JSONObject obj = jArraySlots.getJSONObject(i);
String name = obj.optString("name");
if("singer".equals(name))
singer = obj.optString("value");
else if("songname".equals(name))
songName = obj.optString("value"); }}else if((modifier ! = null) && ("stop".equals(modifier)))
{
if(mBookUtil ! = null)
if(mBookUtil.isPlaying())
mBookUtil.stop();
}else if((modifier ! = null) && ("pause".equals(modifier)))
{
if(mBookUtil ! = null)
if(mBookUtil.isPlaying())
mBookUtil.pause();
}else if((modifier ! = null) && ("resume_play".equals(modifier)))
{
if(mBookUtil ! = null)
mBookUtil.resumePlay();
}else if((modifier ! = null) && ("add_volume".equals(modifier)))
{
if(mBookUtil ! = null)
mBookUtil.addVolume();
}else if((modifier ! = null) && ("del_volume".equals(modifier)))
{
if(mBookUtil ! = null)
mBookUtil.delVolume();
}else if((modifier ! = null) && ("next".equals(modifier)))
{
if(mBookUtil ! = null)
mBookUtil.next();
}else if((modifier ! = null) && ("previous".equals(modifier)))
{
if(mBookUtil ! = null)
mBookUtil.prev();
}else if((modifier ! = null) && ("play_index".equals(modifier)))
{
int position = 0;
if(jArraySlots ! = null)
for(int i=0,k=jArraySlots.length(); i<k; i++)
{
JSONObject obj = jArraySlots.getJSONObject(i);
JSONObject jNumDetial = obj.getJSONObject("num_detail");
String index = jNumDetial.optString("recommend_value");
position = Integer.parseInt(index) - 1;
}
if(mBookUtil ! = null)
mBookUtil.skipTo(position); }}}if(songName ! = null)
{
if(singer ! = null) {}else{
mBookUtil.searchBookAndPlay(songName,0.0); }}else if(singer ! = null)
{
mBookUtil.searchBookAndPlay(songName,0.0);
}
serverMessage = jObj.optJSONObject("desc_obj").opt("result").toString();
}
catch (Exception e)
{
e.printStackTrace();
}
// Send a message to update the voice recognition text
sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT, 0.0.null, input);
// Send a message to update the result string returned by the server
sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE,
0.0.null, serverMessage);
}Copy the codeWith the sentence “I want to hear the Romance of The Three Kingdoms”, the server returns the following data:
{"data": {"asr": {"result": "I want to hear the Romance of The Three Kingdoms.","speech_status": 0,"final": true,"status": 0
},"nli": [{"desc_obj": {"result": "Search efforts underway. Please wait.","status": 0
},"semantic": [{"app": "musiccontrol","input": "I want to hear the Romance of The Three Kingdoms.","slots": [{"name": "songname","value": Romance of The Three Kingdoms }],"modifier": [
"play"
],"customer": "58df512384ae11f0bb7b487e"
}],"type": "musiccontrol"
}]},"status": "ok"
}Copy the code1) The type of nLI type is musicControl, which is the type of app returned by syntax. This online listening demo only cares about the type of app musicControl, ignoring the rest.
3) In a semantic nLI, the input value is the user’s word, the same as the result in ASR. Modifier represents the returned action. Here you can see that Play means play. The data in slots indicates that the song name is romance of three Kingdoms. Movement is play, the content is the name of the song is the romance of The Three Kingdoms, in this demo call mBookUtil. SearchBookAndPlay (songName, 0, 0); Will first query, query results will be sent to play the message to play, I want to listen to the romance of The Three Kingdoms this process is finished.
About online listening see blog: blog.csdn.net/ls0609/arti…
4. Download link of source code
pan.baidu.com/s/1o8OELdC
5. Related links
Voice demo:blog.csdn.net/ls0609/arti charge to an account…
Olami open platform syntax written introduction: blog.csdn.net/ls0609/arti…
Olami open platform syntax: cn.olami.ai/wiki/? Mp = nl…