That whole picture thing

Q: What is the size of a 55KB PNG image with a resolution of 1080 * 480 when it is loaded near memory?

Image memory size

Image memory size = resolution * pixel size

Among them, different data formats have different pixel sizes:

  • ALPHA_8: 1B
  • RGB_565: 2B
  • ARGB_4444: 2B
  • ARGB_8888: 4B
  • RGBA_F16: 8B

Now back to the above problem, in a computer display of 55KB image, PNG is just the container of the image, they go through the corresponding compression algorithm to convert each pixel of the original image into another data format.

In general, the content of this image should be: 1080 * 480 * 4B = 1.98m.

Each device has its own differences. Take Android as an example. If we put the same image in the res/drawable directory of different DPI, it will occupy different memory.

This is because bitmap.decoderesource () on Android does a width/height conversion based on where the image is stored:

Converted height = original height * (DPI of the device/DPI of the directory)

Converted width = original width * (DPI corresponding to the dip/directory of the device)

Suppose your phone’s DPI is 320 (corresponding to xhdPI) and you put the above image in the xhdPI directory:

Image memory = 1080 * (320/320) * 480 * (320/320) * 4B = 1.98m

For the same phone, place the above image in the hdPI (240 dpi) directory:

Image memory = 1080 * (320/240) * 480 * (320/240) * 4B = 3.52 M

To view the density of a mobile phone, run the following command:

adb shell cat system/build.prop|grep density

This command can get the dPI of the phone. Usually, the unit in the layout is DP. How much px is 1 DP?

According to the official conversion formula, 1 dp is equal to 1 px for a 160 Dpi phone, or 2.75 px for a 440 Dpi phone

How to reduce the memory of an image

Bitmap attributes

Let’s take a look at the attributes of BitmapOption:

  • InBitmap — The Bitmap is reused when parsing a Bitmap, but must be large and inMutable
  • InMutable – Specifies whether a Bitmap can be changed, for example, by adding a line segment several pixels apart
  • InJustDecodeBounds – True returns only the width and height properties of the Bitmap
  • InSampleSize — must >=1, indicating the compression ratio of the Bitmap, such as inSampleSize=4, will return a Bitmap 1/16 of the size of the original
  • Bitmap
  • InPreferredConfig – Bitmap. Config. ARGB_8888, etc
  • InDither — Whether to jitter, false by default
  • InPremultiplied — Defaults to true and generally does not change its value
  • InDensity — The pixel density of a Bitmap
  • InTargetDensity — The final pixel density of the Bitmap
  • InScreenDensity — The pixel density of the current screen
  • InScaled — Supports scaling, defaults to true, when set, Bitmap will be scaled at inTargetDensity
  • InPurgeable — Whether the memory used to store the Pixel can be reclaimed if the system runs out of memory
  • InInputShareable — inPurgeable takes effect only when inPurgeable is true. Whether an InputStream can be shared
  • InPreferQualityOverSpeed — true ensures Bitmap quality and decoding speed
  • OutWidth – The width of the returned Bitmap
  • OutHeight – The height of the Bitmap returned
  • InTempStorage — Temporary space for decoding. Recommended 16 x 1024

Lower resolution

The android system can provide the corresponding API in proportion to compress picture BitmapFactory. The Options. InSampleSize inSampleSzie value, the greater the higher compression ratio

Changing the data format

The android system defaults to ARGB_8888, so each pixel should be 4B in size. You can change the format to RGB_565

Glide image compression

A simple process for loading images

The last step we use Glide to load the image is #into(ImageView) and we directly locate the RequestBuilder#into(ImageView) method:

BaseRequestOptions<? > requestOptions =this; .// Build Glide's Scale Type based on ImageView's native Scale Type
    Request = buildRequest(target, targetListener, options) // Singlerequest.obtain () is finally called to create the request
    requestManager.track(target, request); // Request the URL to load the image from here
Copy the code

Targettracker.track (target) is executed in the tarck() method, and this line of code is used to track the lifecycle

Equestlerequest# onResourceReady(Resource
resource, DataSource DataSource) method.

Equestlerequest# onSizeReady and call Engine#load() to download and decode the image.

.// Omit the code to read images from memory and disk respectively
EnginJob<R> engineJob = engineJobFactory.build();
DecodeJob<R> decodeJob = decodeJobFacotry.build();
josbs.put(key, enginJob);
engineJob.addCallback(cb);
engineJob.start(decodeJob); // Start decoding
Copy the code

DecodePath#decodeResourceWithList()

Resource<ResourceType> result = null;
for (int i = 0, size = decoders.size(); i < size; i++) {
    ResourceDecoder<DataType, ResourceType> decoder = decoders.get(i);
    result = decoder.decode(data, width, height, options);
}
return result;

Copy the code

Image decoding

Next, the decoding process of the image is analyzed.

First, we need to understand how decoders come from. Originally, when Glide is initialized, all decoders supported by Glide will be registered in the decoderRegistry. Finally call the ResourceDecoderRegistry#getDecoders() method to get the required decoders:

 public synchronized <T, R> List<ResourceDecoder<T, R>> getDecoders(@NonNull Class<T> dataClass,
      @NonNull Class<R> resourceClass) {
    List<ResourceDecoder<T, R>> result = new ArrayList<>();
    for(String bucket : bucketPriorityList) { List<Entry<? ,? >> entries = decoders.get(bucket);if (entries == null) {
        continue;
      }
      for(Entry<? ,? > entry : entries) {if(entry.handles(dataClass, resourceClass)) { result.add((ResourceDecoder<T, R>) entry.decoder); }}}// TODO: cache result list.

    return result;
  }
Copy the code

There are many ResourceDecoder implementation classes in Glide, as shown in the figure below

Glide according to the picture of the resource type will call different Decoder to decode, now we take the most common scene, load network pictures to illustrate. ByteBufferBitmapDecoder is called to load network image (PNG format).

Whether loading network images or loading local resources, are decoded by ByteBufferBitmapDecoder class

public class ByteBufferBitmapDecoder implements ResourceDecoder<ByteBuffer.Bitmap> {
 private final Downsampler downsampler;

 public ByteBufferBitmapDecoder(Downsampler downsampler) {
   this.downsampler = downsampler;
 }

 @Override
 public boolean handles(@NonNull ByteBuffer source, @NonNull Options options) {
   return downsampler.handles(source);
 }

 @Override
 public Resource<Bitmap> decode(@NonNull ByteBuffer source, int width, int height,
     @NonNull Options options)
     throws IOException {
   InputStream is = ByteBufferUtil.toStream(source);
   returndownsampler.decode(is, width, height, options); }}Copy the code

This class is very simple. The main thing is to call Downsampler#decode.

Downsampler

First, take a look at the decode method, which Downsampler provides externally

  public Resource<Bitmap> decode(InputStream is, int requestedWidth, int requestedHeight,
      Options options, DecodeCallbacks callbacks) throws IOException {
    Preconditions.checkArgument(is.markSupported(), "You must provide an InputStream that supports"
        + " mark()");
	/* Start building bitmpFactory.options */
    byte[] bytesForOptions = byteArrayPool.get(ArrayPool.STANDARD_BUFFER_SIZE_BYTES, byte[].class);
    BitmapFactory.Options bitmapFactoryOptions = getDefaultOptions();
    bitmapFactoryOptions.inTempStorage = bytesForOptions;

    DecodeFormat decodeFormat = options.get(DECODE_FORMAT);
    DownsampleStrategy downsampleStrategy = options.get(DownsampleStrategy.OPTION);
    boolean fixBitmapToRequestedDimensions = options.get(FIX_BITMAP_SIZE_TO_REQUESTED_DIMENSIONS);
    booleanisHardwareConfigAllowed = options.get(ALLOW_HARDWARE_CONFIG) ! =null && options.get(ALLOW_HARDWARE_CONFIG);

    try {
      Bitmap result = decodeFromWrappedStreams(is, bitmapFactoryOptions,
          downsampleStrategy, decodeFormat, isHardwareConfigAllowed, requestedWidth,
          requestedHeight, fixBitmapToRequestedDimensions, callbacks);
      return BitmapResource.obtain(result, bitmapPool);
    } finally{ releaseOptions(bitmapFactoryOptions); byteArrayPool.put(bytesForOptions); }}Copy the code

This method first sets the required parameters for bitmapFactory.options

  1. inTempStorage

    Temp storage to use for decoding. Suggest 16K or so. Glide is used here

  2. decodeFormat

    Decode format, Glide picture mainly for two modes ARGB_8888, RGB_565

  3. fixBitmapToRequestedDimensions

    Defaults to false (not sure what this property means and can’t be set to true)

  4. isHardwareConfigAllowed

    Hardware bitmap

    Disabled by default

        boolean isHardwareConfigSafe =
        dataSource == DataSource.RESOURCE_DISK_CACHE || decodeHelper.isScaleOnlyOrNoTransform();
    Boolean isHardwareConfigAllowed = options.get(Downsampler.ALLOW_HARDWARE_CONFIG);
Copy the code

DecodeFromWrappedStream is used to retrieve bitmaps. The main logic of this method is as follows:

int[] sourceDimensions = getDimensions(is, options, callbacks, bitmapPool); // Get the width and height of the original image
    int targetWidth = requestedWidth == Target.SIZE_ORIGINAL ? sourceWidth : requestedWidth;
    int targetHeight = requestedHeight == Target.SIZE_ORIGINAL ? sourceHeight : requestedHeight;
calculateScaling(); // Set the inSampleSize scale
calculateConfig();
Bitmap downsampled = decodeStream(is, options, callbacks, bitmapPool);
callbacks.onDecodeComplete(bitmapPool, downsampled);
Copy the code

Let’s clarify these sizes first, taking Width as an example

  1. SourceWidth: The width of the original image you downloaded from the network
  2. RequestedWidth: Defaults to the width of the ImageView
  3. TargeWidth: The width of the resulting bitmap

Next, analyze the calculateScaling method

For example, assume the image’s sourceWidth is 1000, targetWidth is 200, sourceHeight is 1200, and targetWidth is 300

final float exactScaleFactor = downsampleStrategy.getScaleFactor(sourceWidth, sourceHeight, targetWidth, targetHeight); // Assuming the downsampling policy is implemented as CenterOutside, exactScaleFactor = 0.25
SampleSizeRounding rounding = downsampleStrategy.getSampleSizeRounding(sourceWidth,
        sourceHeight, targetWidth, targetHeight); / / rouding for QUALITY
int outWidth = round(exactScaleFactor * sourceWidth); //outWidth = 0.25*1000 + 0.5 = 250
int outHeight = round(exactScaleFactor * sourceHeight); // outHeight = 0.25*1200 + 0.5 = 300 
int widthScaleFactor = sourceWidth / outWidth; //widthScaleFactor = 1000/250 = 4
int heightScaleFactor = sourceHeight / outHeight; //heightScalFactor = 1200/300 = 4
int scaleFactor = rounding == SampleSizeRounding.MEMORY //scaleFactor = 4
        ? Math.max(widthScaleFactor, heightScaleFactor)
        : Math.min(widthScaleFactor, heightScaleFactor);
int powerOfTwoSampleSize  = Math.max(1, Integer.highestOneBit(scaleFactor)); //powerOfTowSampleSize = 4 and can only be 1,2,4,8,16...
if (rounding == SampleSizeRounding.MEMORY
          && powerOfTwoSampleSize < (1.f / exactScaleFactor)) {
        powerOfTwoSampleSize = powerOfTwoSampleSize << 1;
      }
}
options.inSampleSize = powerOfTwoSampleSize;
// 这里暂时还不太理解，看算法这里的 inTragetDesity 和 inDensity 的比值永远为 1
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
  options.inTargetDensity = adjustTargetDensityForError(adjustedScaleFactor);
  options.inDensity = getDensityMultiplier(adjustedScaleFactor);
}
if (isScaling(options)) {
  options.inScaled = true;
} else {
  options.inDensity = options.inTargetDensity = 0;
}
Copy the code

Let’s take a quick look at the CenterOutside class. The code is simple:

    public float getScaleFactor(int sourceWidth, int sourceHeight, int requestedWidth,
        int requestedHeight) {
      float widthPercentage = requestedWidth / (float) sourceWidth;
      float heightPercentage = requestedHeight / (float) sourceHeight;
      return Math.max(widthPercentage, heightPercentage);
    }

    @Override
    public SampleSizeRounding getSampleSizeRounding(int sourceWidth, int sourceHeight,
        int requestedWidth, int requestedHeight) {
      return SampleSizeRounding.QUALITY; // The return value is QUALITY and MEMORY, where QUALITY occupies less MEMORY than MEMORY}}Copy the code

Next set additional properties for options by calling calculateConfig

    if (hardwareConfigState.setHardwareConfigIfAllowed(
        targetWidth,
        targetHeight,
        optionsWithScaling,
        format,
        isHardwareConfigAllowed,
        isExifOrientationRequired)) {
      return;
    }

    // Changing configs can cause skewing on 4.1, see Issue #128
    if (format == DecodeFormat.PREFER_ARGB_8888
        || Build.VERSION.SDK_INT == Build.VERSION_CODES.JELLY_BEAN) {
      optionsWithScaling.inPreferredConfig = Bitmap.Config.ARGB_8888;
      return;
    }

    boolean hasAlpha = false;
    try {
      hasAlpha = ImageHeaderParserUtils.getType(parsers, is, byteArrayPool).hasAlpha();
    } catch (IOException e) {
      if (Log.isLoggable(TAG, Log.DEBUG)) {
        Log.d(TAG, "Cannot determine whether the image has alpha or not from header"
            + ", format " + format, e);
      }
    }

    optionsWithScaling.inPreferredConfig =
        hasAlpha ? Bitmap.Config.ARGB_8888 : Bitmap.Config.RGB_565;
    if (optionsWithScaling.inPreferredConfig == Config.RGB_565) {
      optionsWithScaling.inDither = true;
    }
Copy the code

Finally, the ecodeStream method is called, which gets the bitmap object by compressing the image from the android API BitmapFactory#decodeStream

Special attention is that when we use Glide to load the network image, the default is according to the size of the ImageView for a certain proportion, the detailed calculation process has been mentioned above. However, in practice, it is desirable for users to see the original scene, so we can do this

      ImgurGlide.with(vh.imageView)
          .load(image.link)
          .diskCacheStrategy(DiskCacheStrategy.RESOURCE) // Hard disk cache saves the original image
          .override(Target.SIZE_ORIGINAL, Target.SIZE_ORIGINAL) // Override requestSize to avoid bitmap compression
          .into(vh.imageView);
Copy the code

Skia library

In android, BitmapFactory decodeStream call is natvie method, the function is called skia eventually encodeStream function of library to image compression coding. Next, take a look at the SKia library.

Skia is a c++ implemented code library that exists in android as an extension library in the directory external/ Skia /. Overall, Skia is a relatively simple library that provides basic drawing and simple codec functions in Android. In addition, SKia can also be attached to other third party codec libraries or hardware codec libraries, such as libpng and libjpeg. In Android, skia does just that. Under the external/skia/SRC/images folder, there are several Skimagedecoder_xxx. CPP files, which are inherited from the Skimagedecoder.cpp class. The corresponding type of file is decoded using a third-party library, and finally registered with SkTRegistry, as shown below

static SkTRegistry<SkImageDecoder*, SkStream*> gDReg(sk_libjpeg_dfactory);
static SkTRegistry<SkImageDecoder::Format, SkStream*> gFormatReg(get_format_jpeg);
static SkTRegistry<SkImageEncoder*, SkImageEncoder::Type> gEReg(sk_libjpeg_efactory);
Copy the code

The Android code saves images by calling the Java layer functions — Native layer functions — Skia library functions — corresponding to third-party library functions (such as libjpeg).

Glide – Doubling is a quick way to make everything look like rounded corners, Gaussian blur, or black and white.