In the previous article, I described how to use CameraX. This article will examine the source code for the main processes of CameraX. The source version for this analysis is 1.0.0-alpha06, and the latest CameraX release is 1.0.0-alpha10.

reference

Declare it in build.gradle

def camerax_version = "1.0.0 - alpha06"
implementation "androidx.camera:camera-core:${camerax_version}"
implementation "androidx.camera:camera-camera2:${camerax_version}"

def camerax_view_version = "1.0.0 - alpha03"
def camerax_ext_version = "1.0.0 - alpha03"
//other
// If you to use the Camera View class
implementation "androidx.camera:camera-view:$camerax_view_version"
// If you to use Camera Extensions
implementation "androidx.camera:camera-extensions:$camerax_ext_version"
Copy the code

• Camera-core: Camera core library, implementation of design architecture
• Camera-camera2: Camera A2 configuration and operation encapsulation
• Camera-view: a custom CameraView component
• Camera-extensions: apis for accessing device-specific vendor effects such as bokeh, HDR, and other features

Camera -core and Camera – Camera2 are the required libraries, with which you can easily use the functions of the Camera2 API

CameraX structure

Let’s start with the CameraX properties:

private static final CameraX INSTANCE = new CameraX();
final CameraRepository mCameraRepository = new CameraRepository();
private final AtomicBoolean mInitialized = new AtomicBoolean(false);
private final UseCaseGroupRepository mUseCaseGroupRepository = new UseCaseGroupRepository();
private final ErrorHandler mErrorHandler = new ErrorHandler();
private CameraFactory mCameraFactory;
private CameraDeviceSurfaceManager mSurfaceManager;
private UseCaseConfigFactory mDefaultConfigFactory;
private Context mContext;
Copy the code

Here are a few important attributes:

• CameraRepository: Store a list of available cameras
• UseCaseGroupRepository: Every UseCaseGroupLifecycleController UseCaseGroupLifecycleController instance warehouse, is associated with a LifecycleOwner, The LifecycleOwner moderates the common lifecycle shared by all use cases in the group
• CameraFactory: Camera abstract factory, Camera2CameraFactory is a concrete implementation class
• CameraDeviceSurfaceManager: Camera equipment and its corresponding data flow management, the implementation is Camera2DeviceSurfaceManager
• UseCaseConfigFactory: UseCase configuration factory

CameraX uses the UseCase concept to interact with camera devices.

• Preview (Preview)
• ImageCapture (ImageCapture)
• ImageAnalysis

CameraX initialization

Camera2Initializer

CameraX initialization method: init

public static void init(Context context, @NonNull AppConfig appConfig) {
   INSTANCE.initInternal(context, appConfig);
}
Copy the code

Init is initialized via the ContentProvier configuration, which implements the Camera2Initializer class

public final class Camera2Initializer extends ContentProvider {
    private static final String TAG = "Camera2Initializer";

    @Override
    public boolean onCreate(a) {
        Log.d(TAG, "CameraX initializing with Camera2 ...");

        CameraX.init(getContext(), Camera2AppConfig.create(getContext()));
        return false; }... }Copy the code

AndroidMainifest. XML automatically generates the provider configuration, and the OnCreate call to ContentProvider precedes the OnCreate call to Applicantion.

<provider
   android:name="androidx.camera.camera2.impl.Camera2Initializer"
   android:exported="false"
   android:multiprocess="true"
   android:authorities="${applicationId}.camerax-init"
   android:initOrder="100" />
Copy the code

Camera2AppConfig

Create from AppConfig passed in by the init method:

public static AppConfig create(Context context) {
   // Create the camera factory for creating Camera2 camera objects
   CameraFactory cameraFactory = new Camera2CameraFactory(context);

   // Create the DeviceSurfaceManager for Camera2
   CameraDeviceSurfaceManager surfaceManager = new Camera2DeviceSurfaceManager(context);

   // Create default configuration factory
   ExtendableUseCaseConfigFactory configFactory = new ExtendableUseCaseConfigFactory();
   configFactory.installDefaultProvider(
            ImageAnalysisConfig.class, new ImageAnalysisConfigProvider(cameraFactory, context));
   configFactory.installDefaultProvider(
            ImageCaptureConfig.class, new ImageCaptureConfigProvider(cameraFactory, context));
   configFactory.installDefaultProvider(
            VideoCaptureConfig.class, new VideoCaptureConfigProvider(cameraFactory, context));
   configFactory.installDefaultProvider(
            PreviewConfig.class, new PreviewConfigProvider(cameraFactory, context));

   AppConfig.Builder appConfigBuilder =
            new AppConfig.Builder()
                  .setCameraFactory(cameraFactory)
                  .setDeviceSurfaceManager(surfaceManager)
                  .setUseCaseConfigFactory(configFactory);

   return appConfigBuilder.build();
}
Copy the code

Build from AppConfig.Builder, where the default properties in CameraX are initialized. When we talk about a specific UseCase, we will analyze the specific ConfigProvider in detail

CameraX.initInternal

The actual CameraX initialization method: initInternal

private void initInternal(Context context, AppConfig appConfig) {
   if (mInitialized.getAndSet(true)) {
      return;
   }

   mContext = context.getApplicationContext();
   mCameraFactory = appConfig.getCameraFactory(null);
   if (mCameraFactory == null) {
      throw new IllegalStateException(
               "Invalid app configuration provided. Missing CameraFactory.");
   }

   mSurfaceManager = appConfig.getDeviceSurfaceManager(null);
   if (mSurfaceManager == null) {
      throw new IllegalStateException(
               "Invalid app configuration provided. Missing CameraDeviceSurfaceManager.");
   }

   mDefaultConfigFactory = appConfig.getUseCaseConfigRepository(null);
   if (mDefaultConfigFactory == null) {
      throw new IllegalStateException(
               "Invalid app configuration provided. Missing UseCaseConfigFactory.");
   }

   mCameraRepository.init(mCameraFactory);
}
Copy the code

The corresponding instance of mCameraFactory is Camera2CameraFactory, and McAmerarepository.init (mCameraFactory) performs camera-related initialization

CameraRepository.init

public void init(CameraFactory cameraFactory) {
   synchronized (mCamerasLock) {
      try {
            Set<String> camerasList = cameraFactory.getAvailableCameraIds();
            for (String id : camerasList) {
               Log.d(TAG, "Added camera: "+ id); mCameras.put(id, cameraFactory.getCamera(id)); }... }}Copy the code

GetAvailableCameraIds gets the list of available Camera ids, and Camera2CameraFactory getCamera really initializes the Camera

public BaseCamera getCamera(@NonNull String cameraId) {
   Camera camera = new Camera(mCameraManager, cameraId,
            mAvailabilityRegistry.getAvailableCameraCount(), sHandler);
   mAvailabilityRegistry.registerCamera(camera);
   return camera;
}
Copy the code

Registration by CameraAvailabilityRegistry registerCamera method for Camera so far, CameraX related attribute initialization is complete

bindToLifecycle

The CameraX life cycle process, as well as the data transfer process, is explained from the first UseCase preview. In CameraX, PreviewConfig is used to generate preivew. The preview. SetOnPreviewOutputUpdateListener set up surveillance Camera data flow. This series of processes can be achieved, mainly through CameraX. BindToLifecycle to achieve specific processes:

public static void bindToLifecycle(LifecycleOwner lifecycleOwner, UseCase... useCases) {
   Threads.checkMainThread();
   UseCaseGroupLifecycleController useCaseGroupLifecycleController =
            INSTANCE.getOrCreateUseCaseGroup(lifecycleOwner);
   UseCaseGroup useCaseGroupToBind = useCaseGroupLifecycleController.getUseCaseGroup();

   Collection<UseCaseGroupLifecycleController> controllers =
            INSTANCE.mUseCaseGroupRepository.getUseCaseGroups();
   // Check that UseCase is only available on a lifecycle
   for (UseCase useCase : useCases) {
      for (UseCaseGroupLifecycleController controller : controllers) {
            UseCaseGroup useCaseGroup = controller.getUseCaseGroup();
            if(useCaseGroup.contains(useCase) && useCaseGroup ! = useCaseGroupToBind) {throw new IllegalStateException(
                        String.format(
                              "Use case %s already bound to a different lifecycle.", useCase)); }}}//onBind listens for callbacks
   for (UseCase useCase : useCases) {
      useCase.onBind();
   }

   calculateSuggestedResolutions(lifecycleOwner, useCases);

   for (UseCase useCase : useCases) {
      useCaseGroupToBind.addUseCase(useCase);
      for (String cameraId : useCase.getAttachedCameraIds()) {
            attach(cameraId, useCase);
      }
   }

   useCaseGroupLifecycleController.notifyState();
}
Copy the code

UseCaseGroupLifecycleController

Through Lifecycle components, create UseCaseGroupLifecycleController UseCaseGroup controller, start and stop operation

UseCaseGroupLifecycleController useCaseGroupLifecycleController = INSTANCE.getOrCreateUseCaseGroup(lifecycleOwner); .private UseCaseGroupLifecycleController getOrCreateUseCaseGroup(LifecycleOwner lifecycleOwner) {
   return mUseCaseGroupRepository.getOrCreateUseCaseGroup(
            lifecycleOwner, new UseCaseGroupRepository.UseCaseGroupSetup() {
               @Override
               public void setup(UseCaseGroup useCaseGroup) { useCaseGroup.setListener(mCameraRepository); }}); }Copy the code

Create UseCaseGroupLifecycleController through UseCaseGroupRepository

UseCaseGroupLifecycleController getOrCreateUseCaseGroup( LifecycleOwner lifecycleOwner, UseCaseGroupSetup groupSetup) {
   UseCaseGroupLifecycleController useCaseGroupLifecycleController;
   synchronized (mUseCasesLock) {
      // If there is a cache, return it directly, otherwise create it
      useCaseGroupLifecycleController = mLifecycleToUseCaseGroupControllerMap.get(
               lifecycleOwner);
      if (useCaseGroupLifecycleController == null) { useCaseGroupLifecycleController = createUseCaseGroup(lifecycleOwner); groupSetup.setup(useCaseGroupLifecycleController.getUseCaseGroup()); }}returnuseCaseGroupLifecycleController; }...private UseCaseGroupLifecycleController createUseCaseGroup(LifecycleOwner lifecycleOwner) {...// Need to add observer before creating UseCaseGroupLifecycleController to make sure
   // UseCaseGroups can be stopped before the latest active one is started.
   lifecycleOwner.getLifecycle().addObserver(createLifecycleObserver());
   UseCaseGroupLifecycleController useCaseGroupLifecycleController =
            new UseCaseGroupLifecycleController(lifecycleOwner.getLifecycle());
   // Put it into the map cache
   synchronized (mUseCasesLock) {
      mLifecycleToUseCaseGroupControllerMap.put(lifecycleOwner,
               useCaseGroupLifecycleController);
   }
   return useCaseGroupLifecycleController;
}
Copy the code

Create UseCaseGroupLifecycleController and increase the Lifecycle control life cycle:

UseCaseGroupLifecycleController(Lifecycle lifecycle) {
   this(lifecycle, new UseCaseGroup());
}

/** Wraps an existing {@link UseCaseGroup} so it is controlled by lifecycle transitions. */
UseCaseGroupLifecycleController(Lifecycle lifecycle, UseCaseGroup useCaseGroup) {
   this.mUseCaseGroup = useCaseGroup;
   this.mLifecycle = lifecycle;
   / / bind Lifecycle
   lifecycle.addObserver(this);
}

@OnLifecycleEvent(Lifecycle.Event.ON_START)
public void onStart(LifecycleOwner lifecycleOwner) {
   synchronized(mUseCaseGroupLock) { mUseCaseGroup.start(); }}@OnLifecycleEvent(Lifecycle.Event.ON_STOP)
public void onStop(LifecycleOwner lifecycleOwner) {
   synchronized(mUseCaseGroupLock) { mUseCaseGroup.stop(); }}@OnLifecycleEvent(Lifecycle.Event.ON_DESTROY)
public void onDestroy(LifecycleOwner lifecycleOwner) {
   synchronized(mUseCaseGroupLock) { mUseCaseGroup.clear(); }}Copy the code

Add ON_START, ON_STOP, and ON_DESTROY lifecycle listeners to the code above

calculateSuggestedResolutions

Generate the best solution for each UseCase based on the incoming configuration. The following code expands with Preview UseCase, and the other UseCase code has similar logic.

private static void calculateSuggestedResolutions(LifecycleOwner lifecycleOwner, UseCase... useCases) {
   // There will only one lifecycleOwner active. Therefore, only collect use cases belong to
   // same lifecycleOwner and calculate the suggested resolutions..// Collect new use cases for different camera devices
   for (UseCase useCase : useCases) {
      String cameraId = null;
      try {
            cameraId = getCameraWithCameraDeviceConfig(
                  (CameraDeviceConfig) useCase.getUseCaseConfig());
      } catch (CameraInfoUnavailableException e) {
            throw new IllegalArgumentException(
                  "Unable to get camera id for the camera device config.", e); }}...// Get suggested resolutions and update the use case session configuration
   for (String cameraId : newCameraIdUseCaseMap.keySet()) {
      Map<UseCase, Size> suggestResolutionsMap =
               getSurfaceManager()
                        .getSuggestedResolutions(
                              cameraId,
                              originalCameraIdUseCaseMap.get(cameraId),
                              newCameraIdUseCaseMap.get(cameraId));

      for (UseCase useCase : newCameraIdUseCaseMap.get(cameraId)) {
            Size resolution = suggestResolutionsMap.get(useCase);
            Map<String, Size> suggestedCameraSurfaceResolutionMap = new HashMap<>();
            suggestedCameraSurfaceResolutionMap.put(cameraId, resolution);
            // Update the configurationuseCase.updateSuggestedResolution(suggestedCameraSurfaceResolutionMap); }}}Copy the code

Each updateSuggestedResolution UseCase to update the corresponding configuration – > onSuggestedResolutionUpdated

public void updateSuggestedResolution(Map<String, Size> suggestedResolutionMap) { Map<String, Size> resolutionMap = onSuggestedResolutionUpdated(suggestedResolutionMap); . }Copy the code

OnSuggestedResolutionUpdated regarding implementation of different UseCase, here in the Preview, for example

protected Map<String, Size> onSuggestedResolutionUpdated( Map
       
         suggestedResolutionMap)
       ,> {
   // Get the config for the previous configurationPreviewConfig config = (PreviewConfig) getUseCaseConfig(); String cameraId = getCameraIdUnchecked(config); Size resolution = suggestedResolutionMap.get(cameraId); ./ / set the config
   updateConfigAndOutput(config, resolution);
   returnsuggestedResolutionMap; }...private void updateConfigAndOutput(PreviewConfig config, Size resolution) {
   String cameraId = getCameraIdUnchecked(config);
   // Initialize pipeline
   mSessionConfigBuilder = createPipeline(config, resolution);
   attachToCamera(cameraId, mSessionConfigBuilder.build());
   updateOutput(mSurfaceTextureHolder.getSurfaceTexture(), resolution);
}
Copy the code

Preview.createPipeline

Create the Preview pipeline and, through the PreviewConfig configuration, create the corresponding display Surface and SessionConfig

SessionConfig.Builder createPipeline(PreviewConfig config, Size resolution) {
   Threads.checkMainThread();
   SessionConfig.Builder sessionConfigBuilder = SessionConfig.Builder.createFrom(config);

   final CaptureProcessor captureProcessor = config.getCaptureProcessor(null);
   // The Extensions implementation of the extension
   if(captureProcessor ! =null) {
      CaptureStage captureStage = new CaptureStage.DefaultCaptureStage();
      // TODO: To allow user to use an Executor for the processing.. }else {
      final ImageInfoProcessor processor = config.getImageInfoProcessor(null);
      if(processor ! =null) {
            sessionConfigBuilder.addCameraCaptureCallback(new CameraCaptureCallback() {
               @Override
               public void onCaptureCompleted( @NonNull CameraCaptureResult cameraCaptureResult) {
                  super.onCaptureCompleted(cameraCaptureResult);
                  if (processor.process(
                           newCameraCaptureResultImageInfo(cameraCaptureResult))) { notifyUpdated(); }}}); }// The default Surface
      CheckedSurfaceTexture checkedSurfaceTexture = newCheckedSurfaceTexture(resolution); mSurfaceTextureHolder = checkedSurfaceTexture; sessionConfigBuilder.addSurface(checkedSurfaceTexture); }... }Copy the code

Here you can see the familiar flavor, Surface configuration used in Camera2, Session configuration, which will be used later. In CheckedSurfaceTexture, FixedSizeSurfaceTexture is created to display the image.

Preview.updateOutput

Add data listening

void updateOutput(SurfaceTexture surfaceTexture, Size resolution) { PreviewConfig useCaseConfig = (PreviewConfig) getUseCaseConfig(); . PreviewOutput newOutput = PreviewOutput.create(surfaceTexture, resolution, relativeRotation);// Only update the output if something has changed
   if(! Objects.equals(mLatestPreviewOutput, newOutput)) { SurfaceTexture oldTexture = (mLatestPreviewOutput ==null)?null: mLatestPreviewOutput.getSurfaceTexture(); OnPreviewOutputUpdateListener outputListener = getOnPreviewOutputUpdateListener(); .if(outputListener ! =null) {
            mSurfaceDispatched = true; updateListener(outputListener, newOutput); }}}Copy the code

According to the Preview Settings setOnPreviewOutputUpdateListener, access to the corresponding Listener, through updateListener method callback data.

private void updateListener(OnPreviewOutputUpdateListener listener, PreviewOutput output) {... mOutputUpdateExecutor.execute(() -> listener.onUpdated(output)); . }Copy the code

notifyState

Call UseCaseGroupLifecycleController notifyState, activate the UseCase status, the increase in UseCaseGroupLifecycleController lifecycle listening, The musecasegroup.start method is called in the ON_START state.

void notifyState(a) {
   synchronized (mUseCaseGroupLock) {
      if (mLifecycle.getCurrentState().isAtLeast(State.STARTED)) {
            mUseCaseGroup.start();
      }
      for(UseCase useCase : mUseCaseGroup.getUseCases()) { useCase.notifyState(); }}}Copy the code

UseCaseGroup.start

void start(a) {
   synchronized (mListenerLock) {
      if(mListener ! =null) {
            mListener.onGroupActive(this);
      }
      mIsActive = true; }}Copy the code

To start the start state, call the onGroupActive method of CameraRepository:

public void onGroupActive(UseCaseGroup useCaseGroup) {
   synchronized (mCamerasLock) {
      Map<String, Set<UseCase>> cameraIdToUseCaseMap = useCaseGroup.getCameraIdToUseCaseMap();
      for(Map.Entry<String, Set<UseCase>> cameraUseCaseEntry : cameraIdToUseCaseMap.entrySet()) { BaseCamera camera = getCamera(cameraUseCaseEntry.getKey()); attachUseCasesToCamera(camera, cameraUseCaseEntry.getValue()); }}}...private void attachUseCasesToCamera(BaseCamera camera, Set<UseCase> useCases) {
   camera.addOnlineUseCase(useCases);
}
Copy the code

Camera. AddOnlineUseCase associates UseCase with camera.

Camera.addOnlineUseCase

public void addOnlineUseCase(@NonNull final Collection<UseCase> useCases) {
   if (useCases.isEmpty()) {
      return;
   }

   // Attaches the surfaces of use case to the Camera (prevent from surface abandon crash)
   // addOnlineUseCase could be called with duplicate use case, so we need to filter out
   // use cases that are either pending for addOnline or are already online.
   // It's ok for two threads to run here, since It'll do nothing if use case is already
   // pending.
   synchronized (mPendingLock) {
      for (UseCase useCase : useCases) {
            boolean isOnline = isUseCaseOnline(useCase);
            if (mPendingForAddOnline.contains(useCase) || isOnline) {
               continue; } notifyAttachToUseCaseSurfaces(useCase); mPendingForAddOnline.add(useCase); }}... updateCaptureSessionConfig(); resetCaptureSession(/*abortInFlightCaptures=*/false);

   if (mState == InternalState.OPENED) {
      openCaptureSession();
   } else {
      open();
   }

   updateCameraControlPreviewAspectRatio(useCases);
}
Copy the code

In the addOnlineUseCase method, open opens the Camera device.

Camera.open

public void open(a) {...switch (mState) {
      case INITIALIZED:
            openCameraDevice();
            break;
      case CLOSING:
            setState(InternalState.REOPENING);
            // If session close has not yet completed, then the camera is still open. We
            // can move directly back into an OPENED state.
            // If session close is already complete, then the camera is closing. We'll reopen
            // the camera in the camera state callback.
            // If the camera device is currently in an error state, we need to close the
            // camera before reopening, so we cannot directly reopen.
            if(! isSessionCloseComplete() && mCameraDeviceError == ERROR_NONE) { Preconditions.checkState(mCameraDevice ! =null."Camera Device should be open if session close is not complete");
               setState(InternalState.OPENED);
               openCaptureSession();
            }
            break;
      default:
            Log.d(TAG, "open() ignored due to being in state: "+ mState); }}...void openCameraDevice(a) {
   // Check that we have an available camera to open here before attempting
   // to open the camera again.
   if(! mCameraAvailability.isCameraAvailable()) { Log.d(TAG,"No cameras available. Waiting for available camera before opening camera: "
               + mCameraId);
      setState(InternalState.PENDING_OPEN);
      return;
   } else{ setState(InternalState.OPENING); }...// Actually turn on the cameramCameraManager.openCamera(mCameraId, mExecutor, createDeviceStateCallback()); . }Copy the code

Here is the Camera2 preview process

Camera

CameraX encapsulates the standard Camera2 preview process, and these classes are in the CameraX library

CameraDevice.StateCallback

The openCameraDevice stateCallback

final class StateCallback extends CameraDevice.StateCallback {
   @Override
   public void onOpened(CameraDevice cameraDevice) {
      Log.d(TAG, "CameraDevice.onOpened(): " + cameraDevice.getId());
      mCameraDevice = cameraDevice;
      mCameraDeviceError = ERROR_NONE;
      switch (mState) {
            case CLOSING:
            case RELEASING:
               // No session should have yet been opened, so close camera directly here.
               Preconditions.checkState(isSessionCloseComplete());
               mCameraDevice.close();
               mCameraDevice = null;
               break;
            case OPENING:
            case REOPENING:
               setState(InternalState.OPENED);
               openCaptureSession();
               break;
            default:
               throw new IllegalStateException(
                        "onOpened() should not be possible from state: "+ mState); }}... }...void openCaptureSession(a) {... mCaptureSession.open(validatingBuilder.build(), mCameraDevice); . }Copy the code

CaptureSession.open

Create CaptureSession

void open(SessionConfig sessionConfig, CameraDevice cameraDevice)
      throws CameraAccessException, DeferrableSurface.SurfaceClosedException {
   synchronized (mStateLock) {
      switch (mState) {
            case UNINITIALIZED:
               throw new IllegalStateException(
                        "open() should not be possible in state: " + mState);
            case INITIALIZED:
               //Camera session Config is passed in with the surface implementation of TEMPLATE_PREVIEW by defaultList<DeferrableSurface> surfaces = sessionConfig.getSurfaces(); .// Status updatenotifySurfaceAttached(); mState = State.OPENING; . SessionConfigurationCompat sessionConfigCompat =new SessionConfigurationCompat(
                              SessionConfigurationCompat.SESSION_REGULAR,
                              outputConfigList,
                              getExecutor(),
                              comboCallback);

               CaptureRequest captureRequest =
                        Camera2CaptureRequestBuilder.buildWithoutTarget(
                              captureConfigBuilder.build(),
                              cameraDevice);

               if(captureRequest ! =null) {
                  sessionConfigCompat.setSessionParameters(captureRequest);
               }
               // Create CaptureSession, CameraDeviceCompat has different implementations depending on Android versionCameraDeviceCompat.createCaptureSession(cameraDevice, sessionConfigCompat); . }}}Copy the code

In the use of Camera2, CameraDevice’s createCaptureSession can create preview images, and CameraX’s CaptureSession encapsulates these implementations. The SessionConfig passed in CaptureSession. Open is generated when camera2AppConfig. create is created

public static AppConfig create(Context context) {
   // Create the camera factory for creating Camera2 camera objects
   CameraFactory cameraFactory = new Camera2CameraFactory(context);
   / / PreviewConfig configuration
   configFactory.installDefaultProvider(
            PreviewConfig.class, new PreviewConfigProvider(cameraFactory, context));

   AppConfig.Builder appConfigBuilder =
            new AppConfig.Builder()
                  .setCameraFactory(cameraFactory)
                  .setDeviceSurfaceManager(surfaceManager)
                  .setUseCaseConfigFactory(configFactory);

   return appConfigBuilder.build();
}
/ / getConfig PreviewConfigProvider configuration
@Override
public PreviewConfig getConfig(LensFacing lensFacing) {
   PreviewConfig.Builder builder =
            PreviewConfig.Builder.fromConfig(Preview.DEFAULT_CONFIG.getConfig(lensFacing));
   // SessionConfig containing all intrinsic properties needed for Preview
   SessionConfig.Builder sessionBuilder = new SessionConfig.Builder();
   // createCaptureSession in the Preview setting
   sessionBuilder.setTemplateType(CameraDevice.TEMPLATE_PREVIEW);

   // Add options to UseCaseConfig
   builder.setDefaultSessionConfig(sessionBuilder.build());
   builder.setSessionOptionUnpacker(Camera2SessionOptionUnpacker.INSTANCE);

   CaptureConfig.Builder captureBuilder = newCaptureConfig.Builder(); captureBuilder.setTemplateType(CameraDevice.TEMPLATE_PREVIEW); builder.setDefaultCaptureConfig(captureBuilder.build()); builder.setCaptureOptionUnpacker(Camera2CaptureOptionUnpacker.INSTANCE); . }Copy the code

CameraDeviceCompat. CreateCaptureSession CameraCaptureSession callback

final class StateCallback extends CameraCaptureSession.StateCallback {
   @Override
   public void onConfigured(@NonNull CameraCaptureSession session) {
      synchronized (mStateLock) {
            switch (mState) {
               case UNINITIALIZED:
               case INITIALIZED:
               case OPENED:
               case RELEASED:
                  throw new IllegalStateException(
                           "onConfigured() should not be possible in state: " + mState);
               case OPENING:
                  ...
                  // Issue capture request of enableSession if exists.
                  if(mSessionConfig ! =null) {
                        Config implOptions = mSessionConfig.getImplementationOptions();
                        CameraEventCallbacks eventCallbacks = new Camera2Config(
                              implOptions).getCameraEventCallback(
                              CameraEventCallbacks.createEmptyCallback());
                              CameraEventCallback EnableSession callback can be configured
                        List<CaptureConfig> list =
                              eventCallbacks.createComboCallback().onEnableSession();
                        if (!list.isEmpty()) {
                           issueCaptureRequests(setupConfiguredSurface(list));
                        }
                  }
                  //
                  issueRepeatingCaptureRequests();
                  issueBurstCaptureRequest();
                  break; . }}}}Copy the code

CaptureSession.issueRepeatingCaptureRequests

Enable Camera preview

void issueRepeatingCaptureRequests(a) {... CaptureConfig captureConfig = mSessionConfig.getRepeatingCaptureConfig(); .// The override priority for implementation options
   // P1 CameraEventCallback onRepeating options
   // P2 SessionConfig options
   CaptureConfig.Builder captureConfigBuilder = CaptureConfig.Builder.from(captureConfig);

   / / create CaptureRequest
   CaptureRequest captureRequest = Camera2CaptureRequestBuilder.build(
            captureConfigBuilder.build(), mCameraCaptureSession.getDevice(),
            mConfiguredSurfaceMap);
   if (captureRequest == null) {
         Log.d(TAG, "Skipping issuing empty request for session.");
         return;
   }

   // Set the Capture callback
   CameraCaptureSession.CaptureCallback comboCaptureCallback =
            createCamera2CaptureCallback(
                     captureConfig.getCameraCaptureCallbacks(),
                     mCaptureCallback);

   CameraCaptureSessionCompat.setSingleRepeatingRequest(mCameraCaptureSession,
            captureRequest, mExecutor, comboCaptureCallback);
}
Copy the code

CameraCaptureSessionCompat setSingleRepeatingRequest is to distinguish the Android version

private static CameraCaptureSessionCompatImpl chooseImplementation(a) {
   if (Build.VERSION.SDK_INT >= 28) {
      return new CameraCaptureSessionCompatApi28Impl();
   }

   return new CameraCaptureSessionCompatBaseImpl();
}
/ / CameraCaptureSessionCompatBaseImpl, of peace
public int setSingleRepeatingRequest(@NonNull CameraCaptureSession captureSession, @NonNull CaptureRequest request, @NonNull Executor executor, @NonNull CameraCaptureSession.CaptureCallback listener) throws CameraAccessException {
   Preconditions.checkNotNull(captureSession);

   // Wrap the executor in the callback
   CameraCaptureSession.CaptureCallback cb =
            new CameraCaptureSessionCompat.CaptureCallbackExecutorWrapper(executor, listener);

   return captureSession.setRepeatingRequest(
            request, cb, MainThreadAsyncHandler.getInstance());
}
/ / CameraCaptureSessionCompatApi28Impl, some new version of the API changes
public int setSingleRepeatingRequest(@NonNull CameraCaptureSession captureSession, @NonNull CaptureRequest request, @NonNull Executor executor, @NonNull CameraCaptureSession.CaptureCallback listener) throws CameraAccessException {
   Preconditions.checkNotNull(captureSession);

   // Call through directly to executor API
   return captureSession.setSingleRepeatingRequest(request, executor, listener);
}
Copy the code

From Camera opening to preview and reading various configurations, the whole process is now complete. Next, how to take photos is introduced. This process is relatively simple

ImageCapture.takePicture

The process of taking photos:

sendImageCaptureRequest

Create ImageCaptureRequest, set cameraId, targetRatio, callbacks, etc

private void sendImageCaptureRequest( @Nullable Executor listenerExecutor, OnImageCapturedListener listener) {

   String cameraId = getCameraIdUnchecked(mConfig);

   // Get the relative rotation or default to 0 if the camera info is unavailable
   int relativeRotation = 0;
   try {
      CameraInfoInternal cameraInfoInternal = CameraX.getCameraInfo(cameraId);
      relativeRotation =
               cameraInfoInternal.getSensorRotationDegrees(
                        mConfig.getTargetRotation(Surface.ROTATION_0));
   } catch (CameraInfoUnavailableException e) {
      Log.e(TAG, "Unable to retrieve camera sensor orientation.", e);
   }

   Rational targetRatio = mConfig.getTargetAspectRatioCustom(null);
   targetRatio = ImageUtil.rotate(targetRatio, relativeRotation);

   mImageCaptureRequests.offer(
            new ImageCaptureRequest(relativeRotation, targetRatio, listenerExecutor, listener));
   if (mImageCaptureRequests.size() == 1) { issueImageCaptureRequests(); }}Copy the code

takePictureInternal

void issueImageCaptureRequests(a) {
   if (mImageCaptureRequests.isEmpty()) {
      return; } takePictureInternal(); }...// The photo process
private void takePictureInternal(a) {
   // Customize the Future call chain
   FutureChain.from(preTakePicture(state))
      .transformAsync{
         ...
         return ImageCapture.this.issueTakePicture(state);
      })
      .transformAsync{
         ...
         return ImageCapture.this.postTakePicture(state);
      })
      .addCallback(
         ...
         onTakePictureFinish(null); }Copy the code

Customize the whole photo workflow, take a picture by issueTakePicture, postTakePicture is taken successfully, release resources, cancel 3A. Let’s focus on the issueTakePicture process

issueTakePicture

ListenableFuture<Void> issueTakePicture(TakePictureState state) {... getCurrentCameraControl().submitCaptureRequests(captureConfigs); . }Copy the code

Submit Capture requests through CameraControl. The implementation of CameraControl is Camera2CameraControl.

submitCaptureRequests

public void submitCaptureRequests(@NonNull final List<CaptureConfig> captureConfigs) {
   mExecutor.execute(new Runnable() {
      @Override
      public void run(a) { submitCaptureRequestsInternal(captureConfigs); }}); }...void submitCaptureRequestsInternal(final List<CaptureConfig> captureConfigs) {
   mControlUpdateListener.onCameraControlCaptureRequests(captureConfigs);
   / / mControlUpdateListener Camera callback, onCameraControlCaptureRequests truly in the Camera
}
//Camera.java
public void onCameraControlUpdateSessionConfig(@NonNull SessionConfig sessionConfig) { mCameraControlSessionConfig = sessionConfig; updateCaptureSessionConfig(); }...private void updateCaptureSessionConfig(a) {... SessionConfig sessionConfig = validatingBuilder.build(); mCaptureSession.setSessionConfig(sessionConfig); . } Camera Capture SessionConfig, state control via 'CaptureSessionCopy the code

CaptureSession.setSessionConfig

void setSessionConfig(SessionConfig sessionConfig) {
   synchronized (mStateLock) {
      switch (mState) {
            case UNINITIALIZED:
               throw new IllegalStateException(
                        "setSessionConfig() should not be possible in state: " + mState);
            case INITIALIZED:
            case OPENING:
               mSessionConfig = sessionConfig;
               break;
            case OPENED:
               mSessionConfig = sessionConfig;

               if(! mConfiguredSurfaceMap.keySet().containsAll(sessionConfig.getSurfaces())) { Log.e(TAG,"Does not have the proper configured lists");
                  return;
               }

               Log.d(TAG, "Attempting to submit CaptureRequest after setting");
               issueRepeatingCaptureRequests();
               break;
            case CLOSED:
            case RELEASING:
            case RELEASED:
               throw new IllegalStateException(
                        "Session configuration cannot be set on a closed/released session."); }}}Copy the code

If the Camera is in OPENED state, the photo taking process is started

issueRepeatingCaptureRequests

void issueRepeatingCaptureRequests(a) {... CameraCaptureSession.CaptureCallback comboCaptureCallback = createCamera2CaptureCallback( captureConfig.getCameraCaptureCallbacks(), mCaptureCallback); CameraCaptureSessionCompat.setSingleRepeatingRequest(mCameraCaptureSession, captureRequest, mExecutor, comboCaptureCallback); . }Copy the code

CameraCaptureSessionCompat CameraCaptureSessionCompatBaseImpl depending on Android version and CameraCaptureSessionCompatApi28Impl two implementations, Finally, CameraCaptureSession realizes the real photo taking. After the photo is taken, a callback is made through the Listener you set up initially

ImageCapture.createPipeline

In the Preview section, we explained the bindToLifecycle process, where ImageCapture is also a UseCase. In CameraX calculateSuggestedResolutions method, you will call to the UseCase onSuggestedResolutionUpdated method. In ImageCapture onSuggestedResolutionUpdated method, create the photo data through createPipeline callback

SessionConfig.Builder createPipeline(ImageCaptureConfig config, Size resolution) {...// Same as Camera2
   mProcessingImageResultThread = new HandlerThread("OnImageAvailableHandlerThread");
   mProcessingImageResultThread.start();
   mProcessingImageResultHandler = newHandler(mProcessingImageResultThread.getLooper()); . mImageReader.setOnImageAvailableListener(new ImageReaderProxy.OnImageAvailableListener() {
         @Override
         public void onImageAvailable(ImageReaderProxy imageReader) {
            ImageProxy image = null;
            try {
                  image = imageReader.acquireLatestImage();
            } catch (IllegalStateException e) {
                  Log.e(TAG, "Failed to acquire latest image.", e);
            } finally {
                  if(image ! =null) {
                     // Call the head request listener to process the captured image.
                     ImageCaptureRequest imageCaptureRequest;
                     if((imageCaptureRequest = mImageCaptureRequests.peek()) ! =null) {
                        SingleCloseImageProxy wrappedImage = new SingleCloseImageProxy(
                                 image);
                        wrappedImage.addOnImageCloseListener(mOnImageCloseListener);
                        / / ImageCaptureRequest set the Listener
                        imageCaptureRequest.dispatchImage(wrappedImage);
                     } else {
                        // Discard the image if we have no requests.image.close(); } } } } }, mProcessingImageResultHandler); . }Copy the code

ImageReader sets up Camera data to be called and distributed via ImageCaptureRequest’s dispatchImage method

ImageCaptureRequest.dispatchImage

void dispatchImage(final ImageProxy image) {
   try {
         mListenerExecutor.execute(new Runnable() {
            @Override
            public void run(a) {
               Size sourceSize = new Size(image.getWidth(), image.getHeight());
               if (ImageUtil.isAspectRatioValid(sourceSize, mTargetRatio)) {
                     image.setCropRect(
                           ImageUtil.computeCropRectFromAspectRatio(sourceSize,
                                    mTargetRatio));
               }
               // True callbackmListener.onCaptureSuccess(image, mRotationDegrees); }}); }catch (RejectedExecutionException e) {
         Log.e(TAG, "Unable to post to the supplied executor.");

         // Unable to execute on the supplied executor, close the image.image.close(); }}Copy the code

MListener is an encapsulated Listener, implemented in ImageCapture

Listener

The Listener diagram:

 +-----------------------+
 |                       |
 |ImageCapture.          |
 |OnImageCapturedListener|
 |                       |
 +-----------+-----------+
             |
             |
 +-----------v-----------+      +----------------------+
 |                       |      |                      |
 | ImageSaver.           |      | ImageCapture.        |
 | OnImageSavedListener  +------> OnImageSavedListener |
 |                       |      |                      |
 +-----------------------+      +----------------------+
Copy the code

An implementation of OnImageCapturedListener, in which the OnImageSavedListener set by ImageSaver is called back to the uppermost OnImageSavedListener

OnImageCapturedListener imageCaptureCallbackWrapper =
   new OnImageCapturedListener() {
      @Override
      public void onCaptureSuccess(ImageProxy image, int rotationDegrees) {
         CameraXExecutors.ioExecutor()
                  .execute(
                           newImageSaver( image, saveLocation, rotationDegrees, metadata.isReversedHorizontal, metadata.isReversedVertical, metadata.location, executor, imageSavedListenerWrapper)); }... };// ImageSaver is a Runnable, the main implementation of run
final class ImageSaver implements Runnable {...@Override
   public void run(a) {...// Image processing.if(saveError ! =null) {
         postError(saveError, errorMessage, exception);
      } else{ postSuccess(); }}...private void postSuccess(a) {
      mExecutor.execute(new Runnable() {
         @Override
         public void run(a) {
            // The outermost callbackmListener.onImageSaved(mFile); }}); }}Copy the code

The whole photo taking process and data callback is explained. Through the analysis of CameraX Preview and ImageCapture, CameraX to complete the package of Camera2, unified parameter configuration, automatic Resolution calculation, simplify the development of Camera2, and increase the life cycle control, external exposure only simple interface. Using this library, you can implement the complex operations of the previous Camera2 with a few simple lines of code.

reference

• CameraX architecture
• Core Principles Behind CameraX Jetpack Library