This article is from RTC developer community, senior Android engineer Dongyang Wu

This series of articles will share your experience in implementing multi-party video calls based on Agora SDK 2.1.

In the last article, we learned how to use the Agora SDK for one-to-one chat. This article will discuss how to use the Agora SDK for one-to-one chat. The following functions are required:

  1. The chat feature that was implemented in the last article
  2. Display different UI as the number of people joining and the resolution of their phone camera changes, known as “split screen”.
  3. Click on the small window in the split screen to enlarge the chat window

Split screen

According to the preliminary technical research, the best way of split-screen display is to adopt waterfall flow combined with dynamic chat window, which is more convenient to adapt to the changes of UI. Waterfall flow is a popular list layout that presents a jagged multi-column layout on the interface. Let’s implement a waterfall flow:

There are many ways to realize waterfall flow. This paper uses RecyclerView combined with GridLayoutManager to realize it. Let’s start by customizing a RecyclerView and call it GridVideoViewContainer. The core code is as follows:

int count = uids.size();
if(count < = 2) {/ / only local video or chat room only another enclosing setLayoutManager (new LinearLayoutManager (activity. GetApplicationContext (), orientation,false));
} else if(count > 2) {int itemSpanCount = getNearestSqrt(count); this.setLayoutManager(new GridLayoutManager(activity.getApplicationContext(), itemSpanCount, orientation,false));
}

Copy the code

A LinearLayoutManager is used to LinearLayoutManager a one-on-one chat with only one other person in the chat room. In a truly multiplayer chat room, the waterfall stream is implemented using GridLayoutManager, where itemSpanCount is the number of columns in the waterfall stream.

Now that we have a waterfall stream available, we can implement the dynamic chat window: The point of dynamic chat window is that the size of the item is determined by the aspect ratio of the video, so the Adapter and its corresponding layout should be careful not to write the size. The code that controls the size of an item in the Adapter is as follows:

if (force || mItemWidth == 0 || mItemHeight == 0) {
    WindowManager windowManager = (WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE);
    DisplayMetrics outMetrics = new DisplayMetrics();
    windowManager.getDefaultDisplay().getMetrics(outMetrics);

    int count = uids.size();
    int DividerX = 1;
    int DividerY = 1;

    if (count == 2) {
        DividerY = 2;
    } else if (count >= 3) {
        DividerX = getNearestSqrt(count);
        DividerY = (int) Math.ceil(count * 1.f / DividerX);
    }

    int width = outMetrics.widthPixels;
    int height = outMetrics.heightPixels;

    if (width > height) {
        mItemWidth = width / DividerY;
        mItemHeight = height / DividerX;
    } else{ mItemWidth = width / DividerX; mItemHeight = height / DividerY; }}Copy the code

The code above determines the number of columns and rows based on the number of videos, then the width of the video based on the number of columns and screen width, and then the height of the video based on the aspect ratio and width of the video. It also takes into account the phone’s horizontal and vertical display (if (width > height)).

The code for the layout corresponding to the Adapter is as follows:

<RelativeLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/user_control_mask"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical">

    <ImageView
        android:id="@+id/default_avatar"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_centerInParent="true"
        android:visibility="gone"
        android:src="@drawable/icon_default_avatar"
        android:contentDescription="DEFAULT_AVATAR" />

    <ImageView
        android:id="@+id/indicator"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_centerHorizontal="true"
        android:layout_alignParentBottom="true"
        android:layout_marginBottom="@dimen/video_indicator_bottom_margin"
        android:contentDescription="VIDEO_INDICATOR" />

    <LinearLayout
        android:id="@+id/video_info_container"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignParentTop="true"
        android:layout_marginTop="24dp"
        android:layout_marginStart="15dp"
        android:layout_marginLeft="15dp"
        android:visibility="gone"
        android:orientation="vertical">

        <TextView
            android:id="@+id/video_info_metadata"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:singleLine="true"
            style="@style/NotificationUIText" />
    </LinearLayout>

</RelativeLayout>
Copy the code

As you can see, the size attributes in Layout are wrAP_content, which makes it possible for the size of the item to vary with the aspect ratio of the video.

With the split screen layout in place, we can play chat videos on each item.

Play chat videos

In the Agora SDK, the display of a remote video depends only on the user’s UID, so the data source used is simply defined to contain the UID and the corresponding SurfaceView, like this:

 private final HashMap<Integer, SurfaceView> mUidsList = new HashMap<>();
Copy the code

Whenever someone to join our chat channels, can trigger onFirstRemoteVideoDecoded (int uid, int width, int height, int elapsed) method, first the uid is their uid; Next we will create a new SurfaceView for each item and create render views for it. Finally we will add them to the mUidsList we just created and call setupRemoteVideo(VideoCanvas remote) to play the chat video. The complete code for this process is as follows:

@Override
public void onFirstRemoteVideoDecoded(int uid, int width, int height, int elapsed) {
   doRenderRemoteUi(uid);
}

private void doRenderRemoteUi(final int uid) {
   runOnUiThread(new Runnable() {
       @Override
       public void run() {
           if (isFinishing()) {
               return;
           }

           if (mUidsList.containsKey(uid)) {
               return;
           }

           SurfaceView surfaceV = RtcEngine.CreateRendererView(getApplicationContext());
           mUidsList.put(uid, surfaceV);

           boolean useDefaultLayout = mLayoutType == LAYOUT_TYPE_DEFAULT;

           surfaceV.setZOrderOnTop(true);
           surfaceV.setZOrderMediaOverlay(true);

           rtcEngine().setupRemoteVideo(new VideoCanvas(surfaceV, VideoCanvas.RENDER_MODE_HIDDEN, uid));

           if (useDefaultLayout) {
               log.debug("doRenderRemoteUi LAYOUT_TYPE_DEFAULT " + (uid & 0xFFFFFFFFL));
               switchToDefaultVideoView();
           } else {
               int bigBgUid = mSmallVideoViewAdapter == null ? uid : mSmallVideoViewAdapter.getExceptedUid();
               log.debug("doRenderRemoteUi LAYOUT_TYPE_SMALL " + (uid & 0xFFFFFFFFL) + ""+ (bigBgUid & 0xFFFFFFFFL)); switchToSmallVideoView(bigBgUid); }}}); }Copy the code

The above code is identical to the one to one video above, but careful readers may have noticed that we do not put the generated SurfaceView in the interface, which is exactly the difference from the one to one video: We are going to put the SurfaceView in an abstract VideoViewAdapter class. The key code is as follows:

SurfaceView target = user.mView;
VideoViewAdapterUtil.stripView(target);
holderView.addView(target, 0, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
Copy the code

The holderView is the root layout of the ViewHolder layout. See the code at the end of this article for details.

In this way, we can use the split screen mode to play the user chat video in multi-person chat. What if we want to zoom in on a particular user’s video?

Full screen and small window

When a user double-clicks an item, he wants the corresponding video to be displayed in full screen while other videos become small Windows. Then, we define a double-clicking event interface first:

public interface VideoViewEventListener { void onItemDoubleClick(View v, Object item); } concrete realization way is as follows: mGridVideoViewContainer. SetItemEventHandler (newVideoViewEventListener() {
    @Override
    public void onItemDoubleClick(View v, Object item) {
        log.debug("onItemDoubleClick " + v + "" + item + "" + mLayoutType);

        if (mUidsList.size() < 2) {
            return;
        }

        UserStatusData user = (UserStatusData) item;
        int uid = (user.mUid == 0) ? config().mUid : user.mUid;

        if(mLayoutType == LAYOUT_TYPE_DEFAULT && mUidsList.size() ! = 1) { switchToSmallVideoView(uid); }else{ switchToDefaultVideoView(); }}});Copy the code

The method of playing the selected video in full screen is easy to understand, let’s just look at the method of generating the small window list:

private void switchToSmallVideoView(int bigBgUid) {
    HashMap<Integer, SurfaceView> slice = new HashMap<>(1);
    slice.put(bigBgUid, mUidsList.get(bigBgUid));
    Iterator<SurfaceView> iterator = mUidsList.values().iterator();
    while (iterator.hasNext()) {
        SurfaceView s = iterator.next();
        s.setZOrderOnTop(true);
        s.setZOrderMediaOverlay(true);
    }

    mUidsList.get(bigBgUid).setZOrderOnTop(false);
    mUidsList.get(bigBgUid).setZOrderMediaOverlay(false);

    mGridVideoViewContainer.initViewContainer(this, bigBgUid, slice, mIsLandscape);

    bindToSmallVideoView(bigBgUid);

    mLayoutType = LAYOUT_TYPE_SMALL;

    requestRemoteStreamType(mUidsList.size());
}
Copy the code

In addition, everything is the same as normal waterfall stream view, including double clicking on the item in the small window to play it in full screen.

Here we have used Agora SDK to complete a simple multiplayer chat demo with basic functions, there are still a lot of things to do to transition, here is a simple summary of it!

conclusion

Agora provides high-quality video communication SDK, which not only covers mainstream operating systems with high integration efficiency, but also supports multiple modes of video calls including chat, conference, live broadcast and other functions. The API design in THE SDK can basically meet most of the development needs, and hide the underlying development, only need to provide SurfaceView and UID to play the video, so it is very friendly for the developers of the App layer. Ideal for developers with video chat development needs. In the video field entrepreneurship explosion today, I suggest that more developers who want to engage in this field can have a try.

If you have a development problem with this article, please visit the Sound Net Agora q&A section and post to the sound Net engineers.