Many developers in the Android platform RTMP push stream or lightweight RTSP service (camera or the same screen), always feel that the interface is not enough, take The Live SDK as an example (Github) to summarize, we need to support the regular pre-coding audio and video data types:
OnPreviewFrame() : OnPreviewFrame() : OnPreviewFrame() :
@Override
public void onPreviewFrame(byte[] data, Camera camera) {
frameCount++;
if (frameCount % 3000 == 0) {
Log.i("OnPre", "gc+");
System.gc();
Log.i("OnPre", "gc-");
}
if (data == null) {
Parameters params = camera.getParameters();
Size size = params.getPreviewSize();
int bufferSize = (((size.width | 0x1f) + 1) * size.height * ImageFormat.getBitsPerPixel(params.getPreviewFormat())) / 8;
camera.addCallbackBuffer(new byte[bufferSize]);
} else {
if (isRTSPPublisherRunning || isPushingRtmp || isRecording || isPushingRtsp) {
libPublisher.SmartPublisherOnCaptureVideoData(publisherHandle, data, data.length, currentCameraType, currentOrigentation);
}
camera.addCallbackBuffer(data);
}
}
Copy the codeCorresponding interface definition:
/** * Set live video data(no encoded data). * * @param cameraType: CAMERA_FACING_BACK with 0, CAMERA_FACING_FRONT with 1 * * @param curOrg: * PORTRAIT = 1; // LANDSCAPE = 2; * * LANDSCAPE_LEFT_HOME_KEY = 3; / / landscape home button on the left of the @ * * return {0} if successful * / public native int SmartPublisherOnCaptureVideoData (long handle. byte[] data, int len, int cameraType, int curOrg);Copy the code2. Some customized devices only support YV12 data:
/** * @param data: YV12 data ** @param width: width ** @param height: height ** @param y_stride: Y * * @param v_stride: v * * @param u_stride: u * * rotation_degree: * * @return {0} if successful */ public native int SmartPublisherOnYV12Data(long handle, SmartPublisherOnYV12Data) byte[] data, int width, int height, int y_stride, int v_stride, int u_stride, int rotation_degree);Copy the code3. Support NV21 data interface:
Nv21 data interface, in addition to the conventional camera data access, part of the customized camera out of the data flip, this interface also supports.
/** * @param data: NV21 data ** @param len: data length ** @param width: width ** @param height: * * @param uv_stride: y stride * * @param uv_stride: uv stride * * rotation_degree: * * @return {0} if successful */ public native int SmartPublisherOnNV21Data(long handle, byte[] data, int len, int width, int height, int y_stride, int uv_stride, int rotation_degree); /** * @param data: NV21 data ** @param len: data length ** @param width: width ** @param height: * * @param uv_stride: y stride * * @param uv_stride: uv stride * * rotation_degree: Rotate clockwise, must be 0, 90, 180, 270 * * @param is_vertical_flip: Whether to flip vertically, 0 does not flip, 1 flips * * @param is_horizontal_flip: Whether the flip horizontal, 0 not flip, flip * * @ 1 return {0} if successful * / public native int SmartPublisherOnNV21DataV2 (long handle, byte[] data, int len, int width, int height, int y_stride, int uv_stride, int rotation_degree, int is_vertical_flip, int is_horizontal_flip);Copy the code4. Support YUV data access:
/**
* Set live video data(no encoded data).
*
* @param data: I420 data
*
* @param len: I420 data length
*
* @param yStride: y stride
*
* @param uStride: u stride
*
* @param vStride: v stride
*
* @return {0} if successful
*/
public native int SmartPublisherOnCaptureVideoI420Data(long handle, byte[] data, int len, int yStride, int uStride, int vStride);
Copy the code5. Support RGBA data access (support tailored data access, mainly used in the same screen scenario) :
/**
* Set live video data(no encoded data).
*
* @param data: RGBA data
*
* @param rowStride: stride information
*
* @param width: width
*
* @param height: height
*
* @return {0} if successful
*/
public native int SmartPublisherOnCaptureVideoRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height);
/** * Drop clipped RGBA data **@param data: RGBA data
*
* @param rowStride: stride information
*
* @param width: width
*
* @param height: height
*
* @paramClipedLeft: left; ClipedTop:; Clipedwidth: clipped width; ClipedHeight: clipped height; Make sure that the width and height of the cut down are even * *@return {0} if successful
*/
public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);
/**
* Set live video data(no encoded data).
*
* @paramData: ABGR flip vertical * *@param rowStride: stride information
*
* @param width: width
*
* @param height: height
*
* @return {0} if successful
*/
public native int SmartPublisherOnCaptureVideoABGRFlipVerticalData(long handle, ByteBuffer data, int rowStride, int width, int height);
Copy the code6. RGB565 data access (mainly used in the same screen scenario) :
/**
* Set live video data(no encoded data).
*
* @param data: RGB565 data
*
* @param row_stride: stride information
*
* @param width: width
*
* @param height: height
*
* @return {0} if successful
*/
public native int SmartPublisherOnCaptureVideoRGB565Data(long handle,ByteBuffer data, int row_stride, int width, int height);
Copy the code7. Support camera data access (mainly used for camerA2 interface docking) :
/ * * specifically for android. Media. The Image of android. Graphics. ImageFormat. YUV_420_888 format provided interfaces * * @ param width: * * @param height: must be a multiple of 8 * * @param crop_left: Left upper corner of the shear horizontal coordinates, generally based on android. The media. Image. GetCropRect () fill @ * * param crop_top: Shear vertical coordinates of the upper left corner, generally based on android. Media. Image. GetCropRect () fill @ * * param crop_width: Must be in multiples of 8, 0 will ignore this parameter, generally based on android. Media. Image. GetCropRect () fill @ * * param crop_height: Must be a multiple of 8, 0 will ignore this argument, Generally based on android. Media. Image. GetCropRect () fill * * @ param y_plane corresponding android. Media. Image. The Plane [0]. GetBuffer () * * @ param Y_row_stride corresponding android. Media. Image. The Plane [0]. GetRowStride () * * @ param u_plane corresponding android. Media. Image. The Plane [1]. The getBuffer () * * @ param v_plane corresponding android. Media. Image. The Plane [2]. The getBuffer () * * @ param uv_row_stride . Corresponding android. Media. The Image Plane. [1] getRowStride () * * @ param uv_pixel_stride corresponding android. Media. Image. The Plane [1]. The getPixelStride () * * @param rotation_degree: rotate clockwise, must be 0, 90, 180, 270 * * @param is_vertical_flip: @param is_horizontal_flip: Whether to flip horizontally, 0 does not flip, 1 flips * * @param scale_width: Scale width, must be a multiple of 8, 0 is not scaled * * @param scale_height: scale height, must be a multiple of 8, 0 is not scaled * * @param scale_filter_mode: Return {0} if successful */
public native int SmartPublisherOnImageYUV420888(long handle, int width, int height,
int crop_left, int crop_top, int crop_width, int crop_height,
ByteBuffer y_plane, int y_row_stride,
ByteBuffer u_plane, ByteBuffer v_plane, int uv_row_stride, int uv_pixel_stride,
int rotation_degree, int is_vertical_flip, int is_horizontal_flip,
int scale_width, int scale_height, int scale_filter_mode);
Copy the code8. Support PCM data access:
/** * Pass PCM audio data to SDK every 10ms **@paramPcmdata: PCM data, you need to use ByteBuffer. AllocateDirect allocation, ByteBuffer. IsDirect () is true. *@paramSize: PCM data size x@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} is supported. 44100 * is recommended@paramChannel: indicates the channel. The current channel supports single-channel (1) and dual-channel (2). Single-channel (1) * is recommended@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnPCMData(long handle, ByteBuffer pcmdata, int size, int sample_rate, int channel, int per_channel_sample_number);
/** * Pass PCM audio data to SDK every 10ms **@paramPcmdata: PCM data, you need to use ByteBuffer. AllocateDirect allocation, ByteBuffer. IsDirect () is true. *@paramOffset: indicates the offset * of pcmdata@paramSize: PCM data size x@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} is supported. 44100 * is recommended@paramChannel: indicates the channel. The current channel supports single-channel (1) and dual-channel (2). Single-channel (1) * is recommended@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnPCMDataV2(long handle, ByteBuffer pcmdata, int offset, int size, int sample_rate, int channel, int per_channel_sample_number);
/** * Pass PCM audio data to SDK every 10ms **@paramPcm_short_array: PCM data, short is native Endian order *@paramOffset: array offset *@paramLen: Number of array items *@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} is supported. 44100 * is recommended@paramChannel: indicates the channel. The current channel supports single-channel (1) and dual-channel (2). Single-channel (1) * is recommended@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnPCMShortArray(long handle, short[] pcm_short_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);
Copy the code9. Support remote PCM data access and PCM data access after mixing (mainly used for one-to-one interaction) :
/**
* Set far end pcm data
*
* @param pcmdata : 16bit pcm data
* @param sampleRate: audio sample rate
* @param channel: auido channel
* @param per_channel_sample_number: per channel sample numbers
* @param is_low_latency: if with 0, it is not low_latency, if with 1, it is low_latency
* @return {0} if successful
*/
public native int SmartPublisherOnFarEndPCMData(long handle, ByteBuffer pcmdata, int sampleRate, int channel, int per_channel_sample_number, int is_low_latency);
/** * Pass PCM mix audio data to SDK every 10ms **@paramStream_index: currently, only 1 can be passed, other returns an error *@paramPcm_data: PCM data, you need to use ByteBuffer. AllocateDirect allocation, ByteBuffer. IsDirect () is true. *@paramOffset: indicates the offset * of pcmdata@paramSize: PCM data size x@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} * is supported@paramChannels: The current channel supports single (1) and double (2) channels *@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnMixPCMData(long handle, int stream_index, ByteBuffer pcm_data, int offset, int size, int sample_rate, int channels, int per_channel_sample_number);
/** * Pass PCM mix audio data to SDK every 10ms **@paramStream_index: currently, only 1 can be passed, other returns an error *@paramPcm_short_array: PCM data, short is native Endian order *@paramOffset: array offset *@paramLen: Number of array items *@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} * is supported@paramChannels: The current channel supports single (1) and double (2) channels *@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnMixPCMShortArray(long handle, int stream_index, short[] pcm_short_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);
Copy the codeConclusion:
The above is just the data interface before coding. If a product wants to do enough general, it needs too much docking, don’t you think?