This article may be the latest and most complete CameraX interpretation, the length is longer, enjoy slowly.
Our life has been more and more inseparable from the camera, fromselfietolive.scan the codeAnd then toVRAnd so on. Camera advantages and disadvantages naturally become manufacturers competing to chase the field. For app developers, how to quickly drive the camera, provide excellent shooting experience, and optimize the power consumption of the camera has always been the goal of pursuit.
preface
The Camera interface has been deprecated since Android 5.0, so the usual approach is to use its replacement, the Camera2 interface. But with the advent of CameraX, that option is no longer unique.
Let’s first review how the simple requirement of image preview is implemented using the Camera2 interface.
Camera2
Aside from callbacks, exceptions and other additional processing, it still requires multiple steps to implement, which is more tedious. ※ Length causes omission of code to summarize only the steps ※
Graph TD display TextureView control --> Start HandlerThread in advance and get its Handler instance for camera callback --> Make sure your app has camera permissions --> Listen for TextureView available and configure the camera --> Get and save the ID and parameters of the target lens --> reflect the size/zoom ratio/screen orientation parameters to TextureView --> start the camera from CameraManager --> connect the CameraDevice to the Surface in a successful startup callbackThe same image preview using CameraX, the implementation is very concise.
CameraX
The image preview
I could say a dozen lines. Just like Camera2, you need to show a preview of the control PreviewView to the layout and make sure you get camera permissions. The difference is mainly reflected in the camera configuration steps.
private void setupCamera(PreviewView previewView) {
ListenableFuture<ProcessCameraProvider> cameraProviderFuture =
ProcessCameraProvider.getInstance(this);
cameraProviderFuture.addListener(() -> {
try {
mCameraProvider = cameraProviderFuture.get();
bindPreview(mCameraProvider, previewView);
} catch (ExecutionException | InterruptedException e) {
e.printStackTrace();
}
}, ContextCompat.getMainExecutor(this));
}
private void bindPreview(@NonNull ProcessCameraProvider cameraProvider,
PreviewView previewView) {
mPreview = new Preview.Builder().build();
mCamera = cameraProvider.bindToLifecycle(this,
CameraSelector.DEFAULT_BACK_CAMERA, mPreview);
mPreview.setSurfaceProvider(previewView.getSurfaceProvider());
}
Copy the code
The lens switch
If you want to switch lenses, simply bind the CameraSelector example for the target lens to the CameraProvider. We added buttons on the screen to switch shots.
public void onChangeGo(View view) {
if(mCameraProvider ! =null) {
isBack = !isBack;
bindPreview(mCameraProvider, binding.previewView);
}
}
private void bindPreview(@NonNull ProcessCameraProvider cameraProvider,
PreviewView previewView) {... CameraSelector cameraSelector = isBack ? CameraSelector.DEFAULT_BACK_CAMERA : CameraSelector.DEFAULT_FRONT_CAMERA;// Ensure that all bindings are unbound before binding to prevent CameraProvider from being repeatedly bound to Lifecycle and an exception will occur
cameraProvider.unbindAll();
mCamera = cameraProvider.bindToLifecycle(this, cameraSelector, mPreview); . }Copy the code
The lens focus
A shot that is not focused is incomplete, we listen to Preview’s touch event to tell the CameraX to start focusing with the touch coordinates.
protected void onCreate(@Nullable Bundle savedInstanceState) {... binding.previewView.setOnTouchListener((v, event) -> { FocusMeteringAction action =new FocusMeteringAction.Builder(
binding.previewView.getMeteringPointFactory()
.createPoint(event.getX(), event.getY())).build();
try {
showTapView((int) event.getX(), (int) event.getY());
mCamera.getCameraControl().startFocusAndMetering(action);
}...
});
}
private void showTapView(int x, int y) {
PopupWindow popupWindow = new PopupWindow(ViewGroup.LayoutParams.WRAP_CONTENT,
ViewGroup.LayoutParams.WRAP_CONTENT);
ImageView imageView = new ImageView(this);
imageView.setImageResource(R.drawable.ic_focus_view);
popupWindow.setContentView(imageView);
popupWindow.showAsDropDown(binding.previewView, x, y);
binding.previewView.postDelayed(popupWindow::dismiss, 600);
binding.previewView.playSoundEffect(SoundEffectConstants.CLICK);
}
Copy the code
CameraXThese usage scenarios are uniformly abstracted asUseCase, it has four subclasses, which are respectivelyPreview.ImageCapture.ImageAnalysisandVideoCapture. Here’s how they work.
Images taken
Images can be taken with the help of The takePicture() provided by ImageCapture. The storage can be saved to an external storage space. Of course, you need to obtain read and write permission on the external storage space.
private void takenPictureInternal(boolean isExternal) {
final ContentValues contentValues = new ContentValues();
contentValues.put(MediaStore.MediaColumns.DISPLAY_NAME, CAPTURED_FILE_NAME
+ "_" + picCount++);
contentValues.put(MediaStore.MediaColumns.MIME_TYPE, "image/jpeg");
ImageCapture.OutputFileOptions outputFileOptions =
new ImageCapture.OutputFileOptions.Builder(
getContentResolver(),
MediaStore.Images.Media.EXTERNAL_CONTENT_URI, contentValues)
.build();
if(mImageCapture ! =null) {
mImageCapture.takePicture(outputFileOptions, CameraXExecutors.mainThreadExecutor(),
new ImageCapture.OnImageSavedCallback() {
@Override
public void onImageSaved(@NonNull ImageCapture.OutputFileResults outputFileResults) {
Toast.makeText(DemoActivityLite.this."Picture got"+ (outputFileResults.getSavedUri() ! =null
? "@" + outputFileResults.getSavedUri().getPath()
: "") + ".", Toast.LENGTH_SHORT) .show(); }... }); }}private void bindPreview(@NonNull ProcessCameraProvider cameraProvider,
PreviewView previewView) {... mImageCapture =newImageCapture.Builder() .setTargetRotation(previewView.getDisplay().getRotation()) .build(); .// You need to bind the ImageCapture scene together
mCamera = cameraProvider.bindToLifecycle(this, cameraSelector, mPreview, mImageCapture); . }Copy the code
Image analysis
Image analysis refers to real-time analysis of previewed images, identifying the color, content and other information, and applying it in machine learning, QR code recognition and other business scenarios. Continue to make some changes to the demo, add a button to scan the QR code. Click the button to enter the scan code mode, and the result will pop up after the TWO-DIMENSIONAL code is successfully resolved.
public void onAnalyzeGo(View view) {
if (!isAnalyzing) {
mImageAnalysis.setAnalyzer(CameraXExecutors.mainThreadExecutor(), image -> {
analyzeQRCode(image);
});
}
...
}
// Extract the image data from ImageProxy and submit it to the TWO-DIMENSIONAL code framework Zxing for analysis
private void analyzeQRCode(@NonNull ImageProxy imageProxy) {
ByteBuffer byteBuffer = imageProxy.getPlanes()[0].getBuffer();
byte[] data = new byte[byteBuffer.remaining()]; byteBuffer.get(data); . BinaryBitmap bitmap =new BinaryBitmap(new HybridBinarizer(source));
Result result;
try{ result = multiFormatReader.decode(bitmap); }... showQRCodeResult(result); imageProxy.close(); }private void showQRCodeResult(@Nullable Result result) {
if(binding ! =null&& binding.qrCodeResult ! =null) { binding.qrCodeResult.post(() -> binding.qrCodeResult.setText(result ! =null ? "Link:\n" + result.getText() : "")); binding.qrCodeResult.playSoundEffect(SoundEffectConstants.CLICK); }}Copy the code
Video recording
StartRecording () with VideoCapture can record video. Add a button to the demo to switch between image shooting and video recording mode. When switching to video recording mode, bind the UseCase for video shooting to the CameraProvider.
public void onVideoGo(View view) {
bindPreview(mCameraProvider, binding.previewView, isVideoMode);
}
private void bindPreview(@NonNull ProcessCameraProvider cameraProvider,
PreviewView previewView, boolean isVideo) {... mVideoCapture =new VideoCapture.Builder()
.setTargetRotation(previewView.getDisplay().getRotation())
.setVideoFrameRate(25)
.setBitRate(3 * 1024 * 1024)
.build();
cameraProvider.unbindAll();
if (isVideo) {
mCamera = cameraProvider.bindToLifecycle(this, cameraSelector,
mPreview, mVideoCapture);
} else {
mCamera = cameraProvider.bindToLifecycle(this, cameraSelector,
mPreview, mImageCapture, mImageAnalysis);
}
mPreview.setSurfaceProvider(previewView.getSurfaceProvider());
}
Copy the codeAfter clicking the record button, make sure you have external storage and audio rights before you start recording the video.
public void onCaptureGo(View view) {
if (isVideoMode) {
if(! isRecording) {// Check permission first.ensureAudioStoragePermission(REQUEST_STORAGE_VIDEO); }}... }private void ensureAudioStoragePermission(int requestId) {...if (requestId == REQUEST_STORAGE_VIDEO) {
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) ! = PackageManager.PERMISSION_GRANTED || ActivityCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) ! = PackageManager.PERMISSION_GRANTED) { ActivityCompat.requestPermissions(...) ;return; } recordVideo(); }}private void recordVideo(a) {
try {
mVideoCapture.startRecording(
new VideoCapture.OutputFileOptions.Builder(getContentResolver(),
MediaStore.Video.Media.EXTERNAL_CONTENT_URI, contentValues)
.build(),
CameraXExecutors.mainThreadExecutor(),
new VideoCapture.OnVideoSavedCallback() {
@Override
public void onVideoSaved(@NonNull VideoCapture.OutputFileResults outputFileResults) {
// Notify user...}}); }... toggleRecordingStatus(); }private void toggleRecordingStatus(a) {
// Stop recording when toggle to false.
if(! isRecording && mVideoCapture ! =null) { mVideoCapture.stopRecording(); }}Copy the code
episode
A problem was found when implementing the video recording function.
When you click the video record button, if you do not have the audio permission at this point, you will ask for it. An exception will occur even if you get permission to call the shooting interface after that. The log shows that the NPE was raised by the AudioRecorder instance for NULL.
A closer look at the logic shows that the demo is now working by binding the VideoCapture to the CameraProvider when switching to video recording mode. At this point in time, AudioRecorder will not be initialized if audio permissions are not obtained. VideoCapture: AudioRecord object cannot initialized correctly VideoCapture: AudioRecord object cannot initialized correctly
Why does NPE still happen when you get permission to call the VideoCapture?
The internal processing of startRecording() is that the request will be terminated if the AudioRecorder instance is null. It doesn’t matter how many times you call it. In fact, there is logic in the latter segment of this function to fetch the AudioRecorder instance again, but it doesn’t have a chance to execute because of the NPE that happened earlier.
// VideoCapture.java
public void startRecording(
@NonNull OutputFileOptions outputFileOptions, @NonNull Executor executor,
@NonNull OnVideoSavedCallback callback) {...try {
// If the mAudioRecorder is null, NPE will terminate the recording request
mAudioRecorder.startRecording();
} catch (IllegalStateException e) {
postListener.onError(ERROR_ENCODER, "AudioRecorder start fail", e);
return; }... mRecordingFuture.addListener(() -> { ...if(getCamera() ! =null) {
// NPE occurred earlier, so you lose the opportunity to get an AudioRecorder instance here againsetupEncoder(getCameraId(), getAttachedSurfaceResolution()); notifyReset(); } }, CameraXExecutors.mainThreadExecutor()); . }Copy the codeIt is not known if this is a bug in the VideoCapture implementation or if the developers intended it. But it doesn’t make sense to call the recording interface when you already have audio rights and still have NPE.
Can only be avoided at this point, or should it be done?
The binding of VideoCapture is now performed before the audio permission is obtained, which makes it possible for the repeated NPE to occur as described above. So you can get away with this by just getting audio permissions and then binding to VideoCapture.
Anyway, wouldn’t it be better to add the audio permission to the VideoCaptue documentation?
Camera Effects extension
The use of the above scenes alone can not meet the increasingly rich shooting needs, portrait, night shooting, beauty and other camera effects are essential. Fortunately, CameraX supports effect extensions. However, not all devices are compatible with this extension. The details can be found in the device compatibility list on the official website.
There are two main types of extendable effects, one is the PreviewExtender for image preview and the other is the ImageCaptureExtender for image shooting.
Each category contains several typical effects.
- NightPreviewExtender Night shot preview
- BokehPreviewExtender Portrait preview
- BeautyPreviewExtender
- HdrPreviewExtender HDR preview
- AutoPreviewExtender Automatic preview
The implementation of these effects is also very simple to turn on.
private void bindPreview(@NonNull ProcessCameraProvider cameraProvider,
PreviewView previewView, boolean isVideo) {
Preview.Builder previewBuilder = new Preview.Builder();
ImageCapture.Builder captureBuilder = newImageCapture.Builder() .setTargetRotation(previewView.getDisplay().getRotation()); . setPreviewExtender(previewBuilder, cameraSelector); mPreview = previewBuilder.build(); setCaptureExtender(captureBuilder, cameraSelector); mImageCapture = captureBuilder.build(); . }private void setPreviewExtender(Preview.Builder builder, CameraSelector cameraSelector) {
BeautyPreviewExtender beautyPreviewExtender = BeautyPreviewExtender.create(builder);
if (beautyPreviewExtender.isExtensionAvailable(cameraSelector)) {
// Enable the extension if available.beautyPreviewExtender.enableExtension(cameraSelector); }}private void setCaptureExtender(ImageCapture.Builder builder, CameraSelector cameraSelector) {
NightImageCaptureExtender nightImageCaptureExtender = NightImageCaptureExtender.create(builder);
if (nightImageCaptureExtender.isExtensionAvailable(cameraSelector)) {
// Enable the extension if available.nightImageCaptureExtender.enableExtension(cameraSelector); }}Copy the codeUnfortunately, I don’t have Redmi 6A in the list of devices that support OEM extensions, so I can’t show you a sample of successful extensions.
Advanced usage
In addition to the common camera usage scenarios mentioned above, there are other optional configuration methods. Space is not limited to detail, interested people can refer to the official website to try.
- The transformation output
CameraXSupport for converting image data to output, such as applied toAs identificationAfter drawing face block diagram
Developer. The android. Google. Cn/training/ca…
- Use case rotationThe screen may rotate during image shooting and analysis. Learn how to configure it so that it rotates
CameraXReal-time access to the screen direction and rotation Angle to capture the correct image
Developer. The android. Google. Cn/training/ca…
- Configuration options control resolution, autofocus, viewfinder shape Settings and other configuration guidance
Developer. The android. Google. Cn/training/ca…
Use attention
-
Remember to call unbindAll() before calling the CameraProvider’s bindToLifecycle(), otherwise duplicate bound exceptions may occur
-
The Analyze () function of the ImageAnalyzer should call ImageProxy’s close() to release the image immediately after analyzing the image, so that subsequent images can be sent. Otherwise, the callback is blocked. Therefore, attention should also be paid to the time consuming problem of image analysis
-
Do not store a reference to each ImageProxy instance after it is closed, because once close() is called, the images will become illegal
-
ImageAnalysis’s clearAnalyzer() should be called after the ImageAnalysis to tell you not to stream the images to avoid wasting performance
-
Do not forget to obtain audio rights when recording a video scene
Interesting compatibility handling
This is an interesting comment I found in the ImageCapture takePicture() document when I implemented the image shooting function.
Before triggering the image capture pipeline, if the save location is a File or MediaStore, it is first verified to ensure it’s valid and writable.
A File is verified by attempting to open a FileOutputStream to it, whereas a location in MediaStore is validated by ContentResolver#insert() creating a new row in the user defined table, retrieving a Uri pointing to it, then attempting to open an OutputStream to it. The newly created row is ContentResolver#delete() deleted at the end of the verification.
On Huawei devices, this deletion results in the system displaying a notification informing the user that a photo has been deleted. In order to avoid this, validating the image capture save location in MediaStore is skipped on Huawei devices.
If the saved Uri is MediaStore, a line will be inserted to verify that the saved path is valid and writable. The test row is deleted after verification.
However, deleting the row on the Huawei device will trigger a notification to delete the photo. So to avoid confusing the user, CameraX will skip the verification of the path on the Huawei device.
class ImageSaveLocationValidator {
// Will determine whether the device brand is Huawei or Honor, if yes, the verification will be skipped
static boolean isValid(final @NonNull ImageCapture.OutputFileOptions outputFileOptions) {...if (isSaveToMediaStore(outputFileOptions)) {
// Skip verification on Huawei devices
final HuaweiMediaStoreLocationValidationQuirk huaweiQuirk =
DeviceQuirks.get(HuaweiMediaStoreLocationValidationQuirk.class);
if(huaweiQuirk ! =null) {
return huaweiQuirk.canSaveToMediaStore();
}
return canSaveToMediaStore(outputFileOptions.getContentResolver(),
outputFileOptions.getSaveCollection(), outputFileOptions.getContentValues());
}
return true; }... }public class HuaweiMediaStoreLocationValidationQuirk implements Quirk {
static boolean load(a) {
return "HUAWEI".equals(Build.BRAND.toUpperCase())
|| "HONOR".equals(Build.BRAND.toUpperCase());
}
/**
* Always skip checking if the image capture save destination in
* {@link android.provider.MediaStore} is valid.
*/
public boolean canSaveToMediaStore(a) {
return true; }}Copy the codeThe advantage of CameraX
From the CameraX in the Camera2 based on a high degree of packaging and compatibility of a large number of devices, so that the CameraX has many advantages.
- Ease of use Using a encapsulated API can effectively achieve this goal
- Device consistency A consistent development experience is achieved regardless of the version and regardless of the differences in the hardware of the device
- The new camera experience offers the same features as the original camera, including beauty shots, through effects extensions
In this paper, the demo
The demo source code is available on Github for reference.
Github.com/ellisonchan…
conclusion
CameraX was released on August 7, 2019 and has been updated from alpha to beta. CameraX’s commitment to unity can be seen in the interesting Huawei device compatibility handling above.
The latest version is still a beta and needs to be improved, but it is not impossible to put into production.
The framework should be used and suggested so that it can be improved and be a boon to both developers and users.
The resources
CameraXUsage Guidelines:Developer. The android. Google. Cn/training/ca…CameraXHistorical version of:Developer. The android. Google. Cn/jetpack/and…CameraXThe compatibility and effects extension supports devices:Developer. The android. Google. Cn/training/ca…CameraXThe official example of:Github.com/android/cam…