Android — Image processing related

Mat, Bitmap, byte

• Mat turn Bitmap

  public static Bitmap matToBitmap(Mat mat) {
        Bitmap resultBitmap = null;
        if(mat ! =null) {
            Imgproc.cvtColor(mat, mat, Imgproc.COLOR_BGR2RGB);
             //resultBitmap = Bitmap.createBitmap(mat.cols(), mat.rows(), Bitmap.Config.ARGB_8888);
            resultBitmap = Bitmap.createBitmap(mat.width(), mat.height(), Bitmap.Config.ARGB_8888);
            if(resultBitmap ! =null)
                Utils.matToBitmap(mat, resultBitmap,true);
        }
        return resultBitmap;
    }
Copy the code

• Turn Bitmap byte

public static Bitmap bytesToBitmap(byte[] byte) {
        if(byte.length ! =0) {
            return BitmapFactory.decodeByteArray(byte, 0, byte.length);
        } else {
            return null; }}Copy the code

Use Cmake to migrate Dlib libraries in NDK

Blog.csdn.net/u012525096/…

• The environment

Cmake (build tool), NDK (environment)

• Get the dlib source code

Download Dlib source code, the official website address: dlib.net/

• Add a dLIB to the project

X directory, go to that directory, copy the Dlib folder (this file is full of source code)

• Configuration CmakeList

Set (distribution_DIR ${CMAKE_SOURCE_DIR}/.. distribution_DIR ${CMAKE_SOURCE_DIR}/.. /distribution) set(pathToDlib ${CMAKE_SOURCE_DIR}/ SRC /main/) # include(${pathToDlib}/dlib/cmake) Native-my is the name of the library, SHARED is the source code (JNI file written by myself). Add_library (native-my SHARED SRC /main/ CPP /native-test.cpp) # set the location of the output library file set_target_properties(native-my PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${distribution_DIR}/libs/${ANDROID_ABI}) # The library needs to be linked (dependent) to other libraries, packaged together with the name of the first parameter below the linked library. ${CMAKE_SOURCE_DIR} = ${CMAKE_SOURCE_DIR} = ${CMAKE_SOURCE_DIR} Target_link_libraries (native-my dlib::dlib log) # Set Debug/Release compilation options, the Debug version of dlib is slow. SET(CMAKE_BUILE_TYPE "RELEASE") SET(CMAKE_CXX_FLAGS_RELEASE "$ENV{CXXFLAGS} -O3 -Wall") SET(CMAKE_BUILE_TYPE "DEBUG") SET(CMAKE_CXX_FLAGS_DEBUG "-g -Wall")Copy the code

• associated

Associate this cmakelists.txt with build.gradle. Open build.gradle under app module and add the following content:

externalNativeBuild { cmake { arguments '-DANDROID_PLATFORM=android-19', '-DANDROID_TOOLCHAIN=clang', '-DANDROID_STL=c++_shared', '-DCMAKE_BUILD_TYPE=Release .. ' cppFlags "-std=c++11 -O3" } } ndk { abiFilters 'armeabi' }Copy the code

Arguments set cmake platform, compiler, STL version library support, Release mode (something dlib must use). CppFlags sets compilation parameters (some support). NDK. AbiFilters sets the filter of library files compiled for different platforms. It is possible to reduce the APK size by using the platform of the corresponding CPU.

Build. Gradle for Android (along with defaultConfig) :

 externalNativeBuild {
        cmake {
            path 'CMakeLists.txt'
        }
    }
Copy the code

Added jniLibs source folder that can be packaged into APK (alongside defaultConfig)

  sourceSets {
        main {
            jniLibs.srcDirs = ['.. /distribution/libs']}}Copy the code

Build. Gradle looks like this:

apply plugin: 'com.android.application'

android {
    compileSdkVersion 26
    defaultConfig {
        applicationId "com.example.shen.testdlib0102"
        minSdkVersion 19
        targetSdkVersion 26
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                arguments '-DANDROID_PLATFORM=android-19'.'-DANDROID_TOOLCHAIN=clang'.'-DANDROID_STL=c++_shared'.'-DCMAKE_BUILD_TYPE=Release .. '
                cppFlags "-std=c++11 -O3"
            }
        }
        ndk {
            abiFilters 'armeabi'
        }
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild {
        cmake {
            path 'CMakeLists.txt'
        }
    }
    sourceSets {
        main {
            jniLibs.srcDirs = ['.. /distribution/libs']
        }
    }
}

dependencies {
    implementation fileTree(dir: 'libs'.include: ['*.jar'])
    implementation 'com. Android. Support: appcompat - v7:26.1.0'
    implementation 'com. Android. Support. The constraint, the constraint - layout: 1.0.2'
    testImplementation 'junit: junit: 4.12'
    androidTestImplementation 'com. Android. Support. Test: runner: 1.0.1'
    androidTestImplementation 'com. Android. Support. Test. Espresso: espresso - core: 3.0.1'
Copy the code

After build/Rebuild or make Module APP,dlib is used as a static library, native my is compiled into a shared library by JNI written by ourselves, and SO file generated by our JNI code is included in JniLibs.

• The problem

The Android platform does not have the relevant GUI interface display, also does not have the related library, needs to own comment out

• More than one file was found with OS Independent Path ‘XXXXX’

Add the following code to build.gradle android in the offending module (at the same level as defaultConfig) :

packagingOptions {
        pickFirst 'lib/armeabi/libnative-my.so'
    }
Copy the code

The reason due to CmakeLists set up SO the library files in the output directory, in the Build. The specified gradle JniLibs. SrcDir, and due to the middle in the Build/intermediates/cmake file. By default, AS will automatically copy temporary directory library files to APK, but by specifying precompiled libraries, SO library files will duplicate, AS does not know which to copy.

YuvData obtained by camera is converted into Mat method in JNI

• Byte[] format is NV21(yuv420SP)
• Obtain grayscale image:

jbyte * pNV21FrameData = env->GetByteArrayElements(NV21FrameData, 0); // enter yuV data Mat mGray(height, width, CV_8UC1, (unsigned char *)pNV21FrameData); Env ->ReleaseByteArrayElements(NV21FrameData, pNV21FrameData, 0);Copy the code

• Get Rgb color map

jbyte * pBuf = (jbyte*)env->GetByteArrayElements(yuv, 0); Mat image(height + height/2,width,CV_8UC1,(unsigned char *)pBuf); // note that this is height+height/2 Mat mBgr; cvtColor(image, mBgr, CV_YUV2BGR_NV21); env->ReleaseByteArrayElements(yuv, pBuf, 0);Copy the code

Android can obtain images through camera and process them in real time. Different mobile phone camera supports different image formats. You can use getCameraPreviewFormat to get the image encoding format supported by Preview. Android uses NV21 (YUV420SP) image format by default, as most phones support it.

In order to achieve the purpose of real-time processing, most of the yuv data we obtained are directly used in c++ in jni to reduce the process of upper-layer image format conversion. Yuv420 itself is a single-channel image. If only grayscale images need to be obtained in image processing, grayscale images can be generated directly through the Mat constructor in c++

extern "C" jboolean Java_my_project_MyRealTimeImageProcessing_CameraPreview_ImageProcessing( JNIEnv* env, jobject thiz, jint width, jint height, jbyteArray NV21FrameData, jintArray outPixels) { jbyte * pNV21FrameData = env->GetByteArrayElements(NV21FrameData, 0); // enter yuV data Mat mGray(height, width, CV_8UC1, (unsigned char *)pNV21FrameData); Env ->ReleaseByteArrayElements(NV21FrameData, pNV21FrameData, 0); return true; }Copy the code

If the image processing requires color images, it is necessary to first convert nv21 type of data into YUV format, and then yuV into BGR color images, the construction of YUV and direct construction of grayscale map is somewhat different, in YUV420 a pixel corresponding to a Y, a 2*2 small square corresponding to a UV, For all YUV420 images, their y-value arrangement is exactly the same, because only Y images are gray images. YUV420sp and YUV420p data format their UV alignment is completely different in principle. 420p, it puts the U first and then the V, which means that the UVs are continuous. And 420SP it is UV, UV so alternately stored. For the size of a YUV420 in memory:

Y = width*height
U = Y/4
v = Y/4
Copy the code

Therefore, only the data size of Y is needed to obtain the grayscale image, so the width and height in the constructor are the width and height of the image, and the memory length needed to obtain the YUV color image is widthheight3/2

Constructor to add 1/2*height to the image:

JNIEXPORT int JNICALL Java_com_ProjectName_nativecaller_ClassName_readYUV420SP(JNIEnv *env, jclass clz, jbyteArray yuv,jint len,jint height,jint width) { jbyte * pBuf = (jbyte*)env->GetByteArrayElements(yuv, 0); Mat image(height + height/2,width,CV_8UC1,(unsigned char *)pBuf); // note that this is height+height/2 Mat mBgr; cvtColor(image, mBgr, CV_YUV2BGR_NV21); imwrite("/mnt/sdcard/readYuv.jpg",mBgr); env->ReleaseByteArrayElements(yuv, pBuf, 0); return 0; }Copy the code

opencv

AndroidOpenCV Camera preview 90 degree rotation problem

Blog.csdn.net/u010112268/…

Android Opencv 410 integration and JNI environment configuration image processing personal summary

Blog.csdn.net/qq_43566257…

Opencv Error – opencv Error: an Assertion failed (SCN = = 3 | | SCN = = 4) in unknown function, the file… \modules\imgproc\src\color.cpp, line 2433

In OpenCV you use cvtColor(frame, current_gray, CV_RGB2GRAY); CvtColor (frame, current_gray, CV_RGB2GRAY) will also give an error if the program runs until a frame is empty.

– Simply porting Dlib and OpencV to Androd platform for face detection

Blog.csdn.net/a568478312/…

Real-time display of Opencv processing after the Camera image AndroidStudio NDK method

Blog.csdn.net/u010677365/…

OpenCV4Android development Record (3) : digital image foundation and OpenCV development introduction

Blog.csdn.net/AndrExpert/…

OpenCV4 Android calls camera

Blog.csdn.net/chy555chy/a…

Learn Android OpenCV from scratch

Github.com/onlyloveyd/…

Android capture camera preview frame, using OpencV and MediaCodec directly record watermark filter video

Blog.csdn.net/u013147734/…

Android uses cmake to access OpencV

www.jianshu.com/p/410422e8d…

Android based on CMake OpenCV development configuration

www.jianshu.com/p/9f5758c36…