Expand your video playback experience

Foldable devices offer users the possibility to do more with their phones, including innovations such as desktop mode, where the hinges are horizontal when the phone is laid flat and the foldable screen is partially open.

Desktop mode is handy when you don’t want to hold your phone in your hand. It’s great for watching media, making video calls, taking photos and even playing games.

A good example is how the Google Duo team optimized their app to work well on tablets and foldable devices.

△ Duo application before and after optimization

In this article, you’ll learn a simple and efficient way to adapt your application layout when running on a foldable device.

This is a simple example of a media player that automatically adjusts the size to avoid the fold in the middle of the screen and adjusts the position of the playback control components from being embedded in the screen when the screen is fully expanded to being shown as a separate panel when the screen is partially folded. As the video shows:

▽ Desktop mode on Samsung Galaxy Z Fold2 5G phone

* Desktop mode is also known as Flex mode on Samsung Galaxy Z series foldable phones.

preparation

The sample application uses Exoplayer, a popular open source media player library on the Android platform. The following Jetpack components are also used:

  • MotionLayout, which is a subclass of interpret tLayout. MotionLayout combines the flexibility of the parent class with the ability to display smooth animation as the view transitions from one pose to another.
  • ReactiveGuide, an invisible component that automatically changes its position when a SharedValue changes. ReactiveGuide needs to work with the Guideline helper class.
  • WindowManager, a library that helps application developers support new device type parameters and provides a common API for different window characteristics.

To use these libraries, you must add the Google Maven libraries to your project and declare the dependencies:

dependencies { ... // Use the following version number when writing, Exoplayer latest version number can be found in the https://github.com/google/ExoPlayer/releases implementation 'com. Google. Android. Exoplayer: exoplayer - core: 2.14.1' implementation 'com. Google. Android. Exoplayer: exoplayer - UI: 2.14.1' Implementation 'androidx. Constraintlayout: constraintlayout: 2.1.0 - rc01' implementation 'androidx. Window: window: 1.0.0 - beta01'... }Copy the code

layout

First consider the layout of the video player Activity, whose root element is a MotionLayout that contains three child views.

<androidx.constraintlayout.motion.widget.MotionLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:id="@+id/root"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/black"
    app:layoutDescription="@xml/activity_main_scene"
    tools:context=".MainActivity">

    <com.google.android.exoplayer2.ui.PlayerView
        android:id="@+id/player_view"
        android:layout_width="0dp"
        android:layout_height="0dp"
        app:layout_constraintBottom_toTopOf="@+id/fold"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:use_controller="false" />

    <androidx.constraintlayout.widget.ReactiveGuide
        android:id="@+id/fold"
        app:reactiveGuide_valueId="@id/fold"
        app:reactiveGuide_animateChange="true"
        app:reactiveGuide_applyToAllConstraintSets="true"
        android:orientation="horizontal"
        app:layout_constraintGuide_end="0dp"
        android:layout_height="wrap_content"
        android:layout_width="wrap_content" />

    <com.google.android.exoplayer2.ui.PlayerControlView
        android:id="@+id/control_view"
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:background="@color/black"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/fold" />

</androidx.constraintlayout.motion.widget.MotionLayout>
Copy the code

Two of these views come from the Exoplayer suite, and you can use them to specify different layouts for the PlayerView (the interface that displays the media) and the PlayerControlView (the container that plays the controls).

The third view is a ReactiveGuide. It is placed between the other two views and acts as a partition of the other two views in the form of Guideline.

The main PlayerView is limited to always being above the ReactiveGuide. This way, when you move the ReactiveGuide from the bottom to the folded position, the layout transformation occurs.

You may want to limit the play control to the bottom of the ReactiveGuide at all times. This way the control is hidden when the screen is fully expanded and appears at the bottom when the screen is partially folded.

Notice the layout_constraintGuide_end attribute on line 28. This is the value you need to change when you move the guide. Since ReactiveGuide is horizontal, this property refers to the distance from the reference line to the bottom of the parent layout.

Make your app aware of screen folding

Now for the most important part: How do you know when your phone is in desktop mode and where it’s folded?

When, after the completion of initialization WindowManager libraries allow you to collect from function WindowInfoRepository. WindowLayoutInfo () the Flow of data Flow < windowLayoutInfo > monitor layout changes:

override fun onStart(a) {
        super.onStart()
        initializePlayer()
        layoutUpdatesJob = uiScope.launch {
            WindowInfoRepository.windowLayoutInfo
                .collect { newLayoutInfo ->
                    onLayoutInfoChanged(newLayoutInfo)
                }
        }
    }

    override fun onStop(a) {
        super.onStop() layoutUpdatesJob? .cancel() releasePlayer() }Copy the code

If you want to learn how to initialize and release an Exoplayer instance, see Exoplayer Codelab.

Every time you get new layout information, you can query the display characteristics and check whether there are folds or hinges in the current display of the device:

private fun onLayoutInfoChanged(newLayoutInfo: WindowLayoutInfo) {
        if (newLayoutInfo.displayFeatures.isEmpty()) {
            // If no features are available on the current screen, we may be watching from the secondary screen,
            // Non-foldable screen or in a foldable home screen but in split mode.
            centerPlayer()
        } else {
            newLayoutInfo.displayFeatures.filterIsInstance(FoldingFeature::class.java) .firstOrNull { feature -> isInTabletopMode(feature) } ? .let { foldingFeature ->valfold = foldPosition(binding.root, foldingFeature) foldPlayer(fold) } ? : run { centerPlayer() } } }Copy the code

Note that if you don’t want to use Kotlin data streams, starting with version 1.0.0-alpha07, you can use the window-Java tool, which provides a set of Java-friendly apis for registering and unregistering callback functions. Or use the window-RxJava2 and window-RxJava3 tools to use the RXJava-adapted apis.

. When the device is in the level to and FoldingFeature isSeparating () returns true, the device may be in the desktop mode.

If so, you can calculate the relative position of the fold and then move the ReactiveGuide to that position; If the situation is reversed, you can move it to 0 (bottom of the screen).

private fun centerPlayer(a) {
        ConstraintLayout.getSharedValues().fireNewValue(R.id.fold, 0)
        binding.playerView.useController = true // Use an inline screen control
    }

    private fun foldPlayer(fold: Int) {
        ConstraintLayout.getSharedValues().fireNewValue(R.id.fold, fold)
        binding.playerView.useController = false // Use the control located on the bottom side of the screen
    }
Copy the code

When you call fireNewValue this way, the library function changes the value of layout_constraintGuide_end. When the device is fully expanded, the entire screen is used to display the main PlayerView.

Final question: Where should you move ReactiveGuide when the device collapses?

The FoldingFeature object has a method bounds() that gets the bounding rectangle information at the fold in the screen coordinate system.

If you are implementing landscape, most of the time the boundary is represented by a rectangle centered vertically in the screen, as wide as the screen, and as high as the hinge (0 for foldable devices, the distance between the two screens for dual-screen devices).

If your application is in full-screen mode, you can anchor the PlayerView at the top of foldingFeatues.bounds ().top, And fix the ControlView at the bottom of foldingFeatures.bounds ().bottom.

In all other cases (not full screen) you need to consider the space taken up by the navigation bar or other UI components on the screen.

To move the guide, you must specify how far it is from the bottom of the parent layout. One possible implementation of calculating the ReactiveGuide proper location function is as follows:

    /** * returns the position of the fold relative to the layout */
    fun foldPosition(view: View, foldingFeature: FoldingFeature): Int {
        valsplitRect = getFeatureBoundsInWindow(foldingFeature, view) splitRect? .let {return view.height.minus(splitRect.top)
        }

        return 0
    }

    /** * Gets the boundary of the displayFeature transform to the view coordinate system and its current position in the window. * The inner margin is included in the calculation by default. * /
    private fun getFeatureBoundsInWindow(
        displayFeature: DisplayFeature,
        view: View,
        includePadding: Boolean = true
    ): Rect? {
        // The position of the view in the window should be in the same coordinate space as the display feature.
        val viewLocationInWindow = IntArray(2)
        view.getLocationInWindow(viewLocationInWindow)

        // Cut the boundary by intersecting the displayFeature boundary rectangle in the window with the view boundary rectangle.
        val viewRect = Rect(
            viewLocationInWindow[0], viewLocationInWindow[1],
            viewLocationInWindow[0] + view.width, viewLocationInWindow[1] + view.height
        )

        // Include an inner margin if necessary
        if(includePadding) { viewRect.left += view.paddingLeft viewRect.top += view.paddingTop viewRect.right -= view.paddingRight  viewRect.bottom -= view.paddingBottom }val featureRectInView = Rect(displayFeature.bounds)
        val intersects = featureRectInView.intersect(viewRect)

        // Check whether the displayFeature and the target view overlap completely
        if ((featureRectInView.width() == 0 && featureRectInView.height() == 0) | |! intersects ) {return null
        }

        // Shift the display feature coordinates to the start of the view coordinate space
        featureRectInView.offset(-viewLocationInWindow[0], -viewLocationInWindow[1])

        return featureRectInView
 }
Copy the code

conclusion

In this article, you learned how to improve the user experience for media applications on foldable devices by implementing flexible layouts that support desktop mode.

Stay tuned for a follow-up article on guidelines for developing different morphologies!

More resources

  • Exoplayer Codelab: Plays video streams with Exoplayer
  • Desktop mode instance application
  • Designed for foldable devices
  • Build applications for foldable devices
  • Jetpack WindowManager
  • Use MotionLayout to manage motion and widget animations

Please click here to submit your feedback to us, or share your favorite content or questions. Your feedback is very important to us, thank you for your support!