Image compression

background

Imaging system two core: lens + photosensitive components

The first degree of Deda crushes men to death

Mi 11 press conference:

For general full-frame DSLR/microsingle cameras, the effective pixel set by the manufacturer is 24 megapixels cmos area :864mm2 2.78 pixels per square millimeter

Asp-c effective pixels are 7.23 pixels per square millimeter on a 24 megapixel CMOS area of 332.3mm2

Huawei mate40 pro 1/1.28 inch 50 megapixels, commonly used 1000w pixel cmos area 192.11mm2 26 /5.2 pixels per square mm

At full-frame DSLR resolution, huawei mate40 pro should have 5 megapixels, not 50 megapixels.

The mi 10 pro 1/1.33 inch (15.28mmx11.46mm) 100 million pixel cmos area 175.09mm2 on 57 pixels per square millimeter

(1 inch equals 25.4mm)

From the point of view of the value, the mobile phone 1000W pixels to the maximum, the effect should be the same as asp- C 24 million pixels to the maximum effect

But: See the details

Zoom in to 100 percent, and your phone is much worse than your camera. Not in the same league

Compression algorithm/image format

GIF: GIF format, good compatibility, poor picture quality.

PNG: transparent channel. File too large. Lossless compression.

Jpeg: Unbeatable compatibility, very good compression ratio. Lossy compression.

Webp: Google’s main push, didn’t catch on

Guetzli: Google came up with a toy algorithm that compressed JPG at a higher compression ratio, but was extremely memory and CPU intensive and had little industrial value.

Hevc/HEIF: H.265 image compression algorithm, copyright issue. Of the stillborn, only Apple has played several versions.

Avif: H.266 image compression algorithm, all open source. High compression ratio, not afraid of repeated compression. Each giant is in fervent development, very promising.

The quality of compressed

bitmap.compress(Bitmap.CompressFormat.JPEG, quality, outputStream);
Copy the code

Quality: a value of 0-100.

The average cell phone/camera photo quality is 96,98.

To the naked eye, there’s no difference between 80 and 100.

The recommended value is 80 to 90

Used for chatting, sharing moments and so on upload, recommended 70

For running algorithm model upload, see the specific requirements of the algorithm. You can use an 80-90 webP

! [image-20210322174334485](/Users/hss/Library/Application Support/typora-user-images/image-20210322174334485.png)

Zoom in and out of pictures

Sampling down interpolating up

Related algorithms:

  • Monolinear/nearest neighbor: fastest, but possibly distorted, with obvious serrations

  • bilinear

  • Double cubic/double cubic

  • Lanczos

Of these,Android only implements monolinear and bilinear.

In the case of downsampling of direct read streams, only single linear sampling is supported.

4. To downsample:

Zoom in: Interpolate upward

Single linear/bilinear operations in Android:

Decode bitmap from stream:

BitmapFactory.Options options = new BitmapFactory.Options();
// inDensity can be used with inTargetDensity, using the same algorithm as inSampleSize
options.inSampleSize = 2;// Downsampling, only single linear sampling is supported
Bitmap compress = BitmapFactory.decodeFile("/sdcard/test.png", options);
Copy the code

Scale operation of bitmap:

Bitmap bitmap = BitmapFactory.decodeFile("/sdcard/test.png");
Bitmap compress = Bitmap.createScaledBitmap(bitmap, bitmap.getWidth()/2, bitmap.getHeight()/2.true);// The last argument is true to use bilinear,false to use unilinear
Copy the code

Or use the matrix operation:

Bitmap bitmap = BitmapFactory.decodeFile("/sdcard/test.png");
Matrix matrix = new Matrix();
matrix.setScale(0.5 f.0.5 f);
bm = Bitmap.createBitmap(bitmap, 0.0, bit.getWidth(), bit.getHeight(), matrix, true);
// The last argument is true to use bilinear,false to use unilinear
Copy the code

Drawing a bitmap on a view:

Monlinear interpolation is used by default when drawing a Bitmap onto a view larger than the bitmap. The serrations are clear

To use bilinear, override the view onPreDraw(canvas) method and add antialiasing to the canvas:

canvas.setDrawFilter(new PaintFlagsDrawFilter(0,Paint.ANTI_ALIAS_FLAG|Paint.FILTER_BITMAP_FLAG));
Copy the code

Here’s a comparison of a flower magnified four times:

More effects see :QQ Music team share: Android image compression technology in detail (part 2)

On Android, bilinear image scaling is recommended

Jpeg compression algorithm improvement

The process of JPEG compression algorithm is:

Color model transformation (RGB to YUV of bitmap), discrete cosine transform, quantization, Z-shape coding, run coding, Huffman coding

Detailed explanation of compression process

  • Enable Hoffman compression on Android: it has been automatically enabled since 7.0.

  • Adaptive quantization tables: Instead of using the two quantization tables built into the JPEG algorithm, use tuned quantization tables for specific types of images

    JPEG compression technology for adaptive quantization tables

Image quality evaluation

reference

Evaluation of image quality without reference (objectification of subjective evaluation)

Indicators/dimensions:

The quality of the image itself is not concerned here, only the quality of the compression obtained by the quantization table in the process of JPG compression

1 Quality evaluation of single drawing

Quantified the quality associated with the factor

Can be used to avoid repeated compression

Given the image and quantization table, the quantization factor is back-estimated to get an estimate of the image quality.

Available library: github.com/sephiroth74… In the

int jpeg_quality =  exif.getQualityGuess()
Copy the code

Or use separate algorithms in how to get JPEG image quality and predict compressed image size

Extract brightness and chroma quantization matrix of JPEG compression from JPEG (file suffix.jpg) in C language

The test results of the two algorithms are as follows:

If the actual compression quality is above 50, the quality can be accurately judged; if the compression quality is below 50, it cannot be accurately judged.

However, this accuracy also depends on the actual algorithm for JPG compression, and if libjpeg is used, the inverse quantization factor is accurate.

If you’re using MozJPEG, the quality of the reverse is also inaccurate:

For example, using Mozjpeg in Squoosh to compress an image to 75, the calculated quality factor is:

Two quality evaluation indicators available in industry

  • Brightness: Calculate the average brightness or ROI. Absolute value is available.

  • Fuzzy degree (Gaussian fuzzy): OpencV’s Laplace operator calculation can be obtained. Can be used for comparative comparison.

Quality comparison evaluation before and after compression

Butteraugli

The Butteraugli project is an image difference comparison library designed to test an image’s psycho-visual error threshold, the point at which viewers begin to notice a deterioration in image quality. In other words, this project attempts to quantify the distortion of your compressed image.

SSIM Structural SIMilarity

PNSR is generally used for rapid video quality evaluation PSNR and SSIM of image quality evaluation indexes

exif

zh.wikipedia.org/wiki/Exif

Not only JPG images can have exIF, but also WebP

Exchangeable Image File Format (Exif) is a file format specially set for digital camera photos, which can record the attribute information and shooting data of digital photos.

Exif was originally developed by Japan Electronics Industry Development Association in 1996, version 1.0. In 1998, an upgrade to version 2.1 added support for audio files. In March 2002, version 2.2 was published.

Exif can be attached to JPEG, TIFF, RIFF and other files to increase the content of the digital camera shooting information and the version information of the index map or image processing software.

Windows 7 has native support for Exif. You can directly view Exif information by right-clicking on an image to open the menu, clicking properties and switching to the details TAB.

Exif information can be edited at will, so it only serves as a reference.

The Exif message starts with 0xFFE1, and the last two bytes indicate the length of the Exif message. So Exif information is up to 64 kB in size and TIFF format internally.

Reading and writing tools

  • The Android: androidx exifinterface. Media. Exifinterface or Android – Exif – Extended

  • Win /Mac: Command line tool – cross-platform, cross-language: exiftool

    Off-topic: Cross-platform, cross-language use of command line tools

    ​ Mac/windows/linux java,js,Python,c,c++

    Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec("cmd.exe /c dir d:\\");
//exec(String command, String[] envp, File dir)
InputStream inputStream = process.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream,"gb2312"));
String line = null;
while((line = br.readLine()) ! =null) {
	System.out.println(line);
}

// Formally, the command-line tool accepts the string[] argument, similar to the Java main method entry:On the command line: Java -xms128m -xmx256m -jar XXX.jar 
  
public static void main(string[] args){
  //args: Can take various parameters passed in when executing Java commands and do some processing with them.
}

// Wrap command line tools in Java: wrap rich arguments in Builder mode
Copy the code

How to read and write exif in Python without changing the image data area?

Copy the entire file, then read exif into memory, modify exif, then write the modified exif to the copied file, so as not to change the image quality.

Do not use PIL, there are pits, will change the quality of the picture

import piexif
from shutil import copyfile
import exifread

original = "IMG_20200628_111551.jpg"
target = "IMG_20200628_111551-3.jpg"
copyfile(original, target)

exif_dict = piexif.load(original)
exif_dict["0th"][piexif.ImageIFD.Copyright] = "lalalla----"

piexif.insert(piexif.dump(exif_dict), target)

# print
f = open(target, 'rb')
tags = exifread.process_file(f)
print(tags.__len__())
for tag in tags.keys():
    print("{}, {}".format(tag, tags[tag]))
Copy the code

The code base

luban2

thumbnailator

Some applications

App development, upload pictures

When Typora blogs, she uploads pictures

Github.com/JuZiSang/pi…

Further reading

QQ Music team share: Android image compression technology in detail (part 1)

QQ Music team share: Android image compression technology details (part 2)

GIF image compression