This is the 11th day of my participation in the August More Text Challenge. For details, see:August is more challenging

background

The purpose of this article is to understand how QML MediaPlayer interacts with Android MediaPlayer. QML MediaPlayer’s play, pause and other functions are passed down. And how android MediaPlayer’s callback functions respond to QML’s slot functions. It is recommended to download the source code of qt5.15 and read it with source insight.

Source process (QT side)

1. We use the MediaPlayer component in QML and set the properties. We add the play and pause methods, and override onPlaying and onStatusChanged. The following code:

MediaPlayer { id: player source: url autoLoad: true autoPlay: false loops: 0 volume: ivolume onPlaying: { console.log("MediaPlayer onPlaying") } onError: { if (MediaPlayer.NoError ! = error) { console.log("[qmlvideo] error " + player.error + " errorString " + player.errorString) } } onStatusChanged: { console.log("onStatusChanged" + status) } } function play() { player.play() }Copy the code

QML QMediaPlayer(D:\WorkSoftware\Qt5.15.2\5.15.2\Src\qtmultimedia\ Src\ multimedia\ PLAYBACK \ QMediaPlayer) CPP), look at the constructor:

QMediaPlayer::QMediaPlayer(QObject *parent, QMediaPlayer::Flags flags): QMediaObject(*new QMediaPlayerPrivate, parent, playerService(flags)) { Q_D(QMediaPlayer); d->provider = QMediaServiceProvider::defaultServiceProvider(); if (d->service == nullptr) { d->error = ServiceMissingError; } else { d->control = qobject_cast<QMediaPlayerControl*>(d->service->requestControl(QMediaPlayerControl_iid)); #ifndef QT_NO_BEARERMANAGEMENT QT_WARNING_PUSH QT_WARNING_DISABLE_DEPRECATED d->networkAccessControl = qobject_cast<QMediaNetworkAccessControl*>(d->service->requestControl(QMediaNetworkAccessControl_iid)); QT_WARNING_POP #endif if (d->control ! = nullptr) { connect(d->control, SIGNAL(mediaChanged(QMediaContent)), SLOT(_q_handleMediaChanged(QMediaContent))); connect(d->control, SIGNAL(stateChanged(QMediaPlayer::State)), SLOT(_q_stateChanged(QMediaPlayer::State))); connect(d->control, SIGNAL(mediaStatusChanged(QMediaPlayer::MediaStatus)), SLOT(_q_mediaStatusChanged(QMediaPlayer::MediaStatus))); connect(d->control, SIGNAL(error(int,QString)), SLOT(_q_error(int,QString))); connect(d->control, &QMediaPlayerControl::durationChanged, this, &QMediaPlayer::durationChanged); connect(d->control, &QMediaPlayerControl::positionChanged, this, &QMediaPlayer::positionChanged); connect(d->control, &QMediaPlayerControl::audioAvailableChanged, this, &QMediaPlayer::audioAvailableChanged); connect(d->control, &QMediaPlayerControl::videoAvailableChanged, this, &QMediaPlayer::videoAvailableChanged); connect(d->control, &QMediaPlayerControl::volumeChanged, this, &QMediaPlayer::volumeChanged); connect(d->control, &QMediaPlayerControl::mutedChanged, this, &QMediaPlayer::mutedChanged); connect(d->control, &QMediaPlayerControl::seekableChanged, this, &QMediaPlayer::seekableChanged); connect(d->control, &QMediaPlayerControl::playbackRateChanged, this, &QMediaPlayer::playbackRateChanged); connect(d->control, &QMediaPlayerControl::bufferStatusChanged, this, &QMediaPlayer::bufferStatusChanged); d->state = d->control->state(); d->status = d->control->mediaStatus(); .Copy the code

Here we just need to focus on the D -> Control object and its bound slot. Here we call a control object with a pointer to D. The pointer to d is a QMediaPlayerPrivate object. QT implements binary compatibility by putting class properties and method implementations in the source file as private Pointers. This hides implementation details and interface changes (Google QT’s D and P Pointers).

2, then qt to Android process tracking take the play method as an example.

void QMediaPlayer::play() { Q_D(QMediaPlayer); if (d->control == nullptr) { QMetaObject::invokeMethod(this, "_q_error", Qt::QueuedConnection, Q_ARG(int, QMediaPlayer::ServiceMissingError), Q_ARG(QString, tr("The QMediaPlayer object does not have a valid service"))); return; }... d->control->play();Copy the code

You can see that the play method on the control object with the d pointer is called. So let’s take a look at the QMediaPlayerPrivate private pointer class:

class QMediaPlayerPrivate : public QMediaObjectPrivate { Q_DECLARE_NON_CONST_PUBLIC(QMediaPlayer) public: QMediaPlayerPrivate() : provider(nullptr) , control(nullptr) , audioRoleControl(nullptr) , customAudioRoleControl(nullptr) , playlist(nullptr) #ifndef QT_NO_BEARERMANAGEMENT , networkAccessControl(nullptr) #endif , state(QMediaPlayer::StoppedState) , status(QMediaPlayer::UnknownMediaStatus) {} QVideoSurfaceOutput surfaceOutput; QMediaContent qrcMedia; QScopedPointer<QFile> qrcFile; QMediaContent rootMedia; QMediaContent pendingPlaylist; QMediaPlayer::State state; QMediaPlayer::MediaStatus status; QMediaPlayer::Error error; int ignoreNextStatusChange; int nestedPlaylists; bool hasStreamPlaybackFeature; QMediaPlaylist *parentPlaylist(QMediaPlaylist *pls); bool isInChain(const QUrl &url); void setMedia(const QMediaContent &media, QIODevice *stream = nullptr); void setPlaylist(QMediaPlaylist *playlist); void setPlaylistMedia(); void loadPlaylist(); void disconnectPlaylist(); void connectPlaylist(); void _q_stateChanged(QMediaPlayer::State state); void _q_mediaStatusChanged(QMediaPlayer::MediaStatus status); void _q_error(int error, const QString &errorString); .Copy the code

And you can see that there’s a control object, and that’s what we saw up there, d-> Control. The control class is located in D: \ WorkSoftware \ Qt5.15.2\5.15.2 \ Src \ qtmultimedia \ Src \ plugins \ android \ Src \ mediaplayer \ qandroidmediaplayercontro As the name implies, it means the control class of Android MediaPlayer. Take a look at the constructor and track its play method:

QAndroidMediaPlayerControl::QAndroidMediaPlayerControl(QObject *parent) : QMediaPlayerControl(parent), mMediaPlayer(new AndroidMediaPlayer), mCurrentState(QMediaPlayer::StoppedState), mCurrentMediaStatus(QMediaPlayer::NoMedia), mMediaStream(0), mVideoOutput(0), mSeekable(true), mBufferPercent(-1), mBufferFilled(false), mAudioAvailable(false), mVideoAvailable(false), mBuffering(false), mState(AndroidMediaPlayer::Uninitialized), mPendingState(-1), mPendingPosition(-1), mPendingSetMedia(false), mPendingVolume(-1), mPendingMute(-1), mReloadingMedia(false), mActiveStateChangeNotifiers(0), MPendingPlaybackRate (1.0), mHasPendingPlaybackRate (false) {connect (mMediaPlayer, SIGNAL (bufferingChanged (qint32)), this,SLOT(onBufferingChanged(qint32))); connect(mMediaPlayer,SIGNAL(info(qint32,qint32)), this,SLOT(onInfo(qint32,qint32))); connect(mMediaPlayer,SIGNAL(error(qint32,qint32)), this,SLOT(onError(qint32,qint32))); connect(mMediaPlayer,SIGNAL(stateChanged(qint32)), this,SLOT(onStateChanged(qint32))); connect(mMediaPlayer,SIGNAL(videoSizeChanged(qint32,qint32)), ............................................... void QAndroidMediaPlayerControl::play() { StateChangeNotifier notifier(this); // We need to prepare the mediaplayer again. if ((mState & AndroidMediaPlayer::Stopped) && ! mMediaContent.isNull()) { setMedia(mMediaContent, mMediaStream); } if (! mMediaContent.isNull()) setState(QMediaPlayer::PlayingState); if ((mState & (AndroidMediaPlayer::Prepared | AndroidMediaPlayer::Started | AndroidMediaPlayer::Paused | AndroidMediaPlayer::PlaybackCompleted)) == 0) { mPendingState = QMediaPlayer::PlayingState; return; } mMediaPlayer->play(); }Copy the code

In the code above, one binds an AndroidMediaPlayer to the signal slot of the Control class, and the other, as you can see from the setState function, maintains a set of playback states for AndroidMediaPlayer in the Control class.

Another interesting thing in this class is StateChangeNotifier notifier(this); This code, in fact, is the use of the class structure and stack-destruct, in which to do things, you can see by the way:

class StateChangeNotifier { public: StateChangeNotifier(QAndroidMediaPlayerControl *mp) : mControl(mp) , mPreviousState(mp->state()) , mPreviousMediaStatus(mp->mediaStatus()) { ++mControl->mActiveStateChangeNotifiers; } ~StateChangeNotifier() { if (--mControl->mActiveStateChangeNotifiers) return; if (mPreviousMediaStatus ! = mControl->mediaStatus()) Q_EMIT mControl->mediaStatusChanged(mControl->mediaStatus()); if (mPreviousState ! = mControl->state()) Q_EMIT mControl->stateChanged(mControl->state()); } private: QAndroidMediaPlayerControl *mControl; QMediaPlayer::State mPreviousState; QMediaPlayer::MediaStatus mPreviousMediaStatus; };Copy the code

Q_EMIT stateChanged during destruction. Q_EMIT stateChanged during destruction. Q_EMIT stateChanged during destruction. Q_EMIT stateChanged during destruction. QMediaPlayer Updates to the final status are sent to QMediaPlayer and eventually to the QML layer.

MMediaPlayer ->play(); MMediaPlayer is an AndroidMediaPlayer object, So go to the AndroidMediaPlayer class (located in D:\WorkSoftware\Qt5.15.2\5.15.2\Src\qtmultimedia\ Src\ plugins\android\ Src\ wrappers\jni\ Android Mediaplayer.cpp) :

static const char QtAndroidMediaPlayerClassName[] = "org/qtproject/qt5/android/multimedia/QtAndroidMediaPlayer"; typedef QVector<AndroidMediaPlayer *> MediaPlayerList; Q_GLOBAL_STATIC(MediaPlayerList, mediaPlayers) Q_GLOBAL_STATIC(QReadWriteLock, rwLock) QT_BEGIN_NAMESPACE AndroidMediaPlayer::AndroidMediaPlayer() : QObject() { QWriteLocker locker(rwLock); auto context = QtAndroidPrivate::activity() ? QtAndroidPrivate::activity() : QtAndroidPrivate::service(); const jlong id = reinterpret_cast<jlong>(this); mMediaPlayer = QJNIObjectPrivate(QtAndroidMediaPlayerClassName, "(Landroid/content/Context; J)V", context, id); mediaPlayers->append(this); } AndroidMediaPlayer::~AndroidMediaPlayer() { QWriteLocker locker(rwLock); const int i = mediaPlayers->indexOf(this); Q_ASSERT(i ! = 1); mediaPlayers->remove(i); } void AndroidMediaPlayer::play() { mMediaPlayer.callMethod<void>("start"); } static void onStateChangedNative(JNIEnv *env, jobject thiz, jint state, jlong id) { Q_UNUSED(env); Q_UNUSED(thiz); QReadLocker locker(rwLock); const int i = mediaPlayers->indexOf(reinterpret_cast<AndroidMediaPlayer *>(id)); if (Q_UNLIKELY(i == -1)) return; Q_EMIT (*mediaPlayers)[i]->stateChanged(state); } bool AndroidMediaPlayer::initJNI(JNIEnv *env) { jclass clazz = QJNIEnvironmentPrivate::findClass(QtAndroidMediaPlayerClassName, env); static const JNINativeMethod methods[] = { {"onErrorNative", "(IIJ)V", reinterpret_cast<void *>(onErrorNative)}, {"onBufferingUpdateNative", "(IJ)V", reinterpret_cast<void *>(onBufferingUpdateNative)}, {"onProgressUpdateNative", "(IJ)V", reinterpret_cast<void *>(onProgressUpdateNative)}, {"onDurationChangedNative", "(IJ)V", reinterpret_cast<void *>(onDurationChangedNative)}, {"onInfoNative", "(IIJ)V", reinterpret_cast<void *>(onInfoNative)}, {"onVideoSizeChangedNative", "(IIJ)V", reinterpret_cast<void *>(onVideoSizeChangedNative)}, {"onStateChangedNative", "(IJ)V", reinterpret_cast<void *>(onStateChangedNative)} }; if (clazz && env->RegisterNatives(clazz, methods, sizeof(methods) / sizeof(methods[0])) ! = JNI_OK) { return false; } return true; }Copy the code

You can see that the AndroidMediaPlayer class also has an mMediaPlayer object, which is a QJNIObjectPrivate object, which represents a JNI object. Can be seen in the constructor: it refers to the org/qtproject qt5 / android/multimedia/QtAndroidMediaPlayer the Java class, look at QtAndroidMediaPlayer class later.

First: You can see that the call to the play method ends up calling the Start function of the Java class through JNI. This is where you get the QT play method on the Android side.

Second: Looking at the static MediaPlayerList from the second and third lines above, it caches the AndroidMediaPlayer pointer into a vector. Take the onStateChangedNative function as an example. This is a native function that is called by the Java layer. Triggered when onStateChanged on Android is received. So the ID in the constructor above is passed to the Android layer first, and then returned when native calls are received, to find the corresponding AndroidMediaPlayer object based on the ID. With the signal slot at the end of step 2 above and the signal slot here, you can see how Android callbacks are passed to QT and even to QML.

Source code Process (Android side)

The Android code is in the JAR package, in D:\WorkSoftware\Qt5.15.2\5.15.2\ Android \jar path.

A quick look at its member objects and some important methods:

public class QtAndroidMediaPlayer { // Native callback functions for MediaPlayer native public void onErrorNative(int what, int extra, long id); native public void onBufferingUpdateNative(int percent, long id); native public void onProgressUpdateNative(int progress, long id); native public void onDurationChangedNative(int duration, long id); native public void onInfoNative(int what, int extra, long id); native public void onVideoSizeChangedNative(int width, int height, long id); native public void onStateChangedNative(int state, long id); private MediaPlayer mMediaPlayer = null; private AudioAttributes mAudioAttributes = null; private HashMap<String, String> mHeaders = null; private Uri mUri = null; private final long mID; public QtAndroidMediaPlayer(final Context context, final long id) { mID = id; mContext = context; } private class MediaPlayerCompletionListener implements MediaPlayer.OnCompletionListener { @Override public void onCompletion(final MediaPlayer mp) { Log.d( TAG,"onCompletion mMediaPlayer.release()"); setState(org.qtproject.qt5.android.multimedia.QtAndroidMediaPlayer.State.PlaybackCompleted); } } private void setState(int state) { if (mState == state) return; mState = state; onStateChangedNative(mState, mID); } public void start() { if ((mState & (org.qtproject.qt5.android.multimedia.QtAndroidMediaPlayer.State.Prepared | org.qtproject.qt5.android.multimedia.QtAndroidMediaPlayer.State.Started | org.qtproject.qt5.android.multimedia.QtAndroidMediaPlayer.State.Paused | org.qtproject.qt5.android.multimedia.QtAndroidMediaPlayer.State.PlaybackCompleted)) == 0) { return; } try { mMediaPlayer.start() setState(org.qtproject.qt5.android.multimedia.QtAndroidMediaPlayer.State.Started); Log.d(TAG, "start"); } catch (final IllegalStateException e) { Log.d(TAG, "" + e.getMessage()); }}Copy the code

There are several things we can see from the above code: First, as mentioned above, both the QT and Android sides maintain a set of states. The second: After the start method is executed, the setState will call the native function onStateChangedNative. In fact, almost every function will setState, one is to update the state here, the other is to call the QT library native function, Update the state in the Control class.

Normally, the jar code is not allowed to change, but for some customization, we can change it. However, we do not need to change the JAR package at this time, because QT contains the code of this JAR package in the source code. Under the \5.15.2\Src\qtmultimedia\ Src\ plugins\android\jar\ Src\ org\qtproject\qt5\android\multimedia, you can add it directly to the project.