Sampling, zooming, panning, chunking, parallel and progressive loading of large image processing in Android.
1. Picture loading basics
1.1. Parameter Meanings
The image loading process involves
dpi: screen pixel density, the number of pixels per inch, base density is160dpi.density: density, proportion value, equivalent todpi/160.dp: density independent pixel unit, display effect on all screens,1dpThe equivalent of160dpiA pixel on the screen.px: Actual pixel unit, equivalent todp * density.
The data format stored by pixels is represented by bitmap. config, represented by ARGB, and A represents transparency. R stands for red; G stands for green; B is blue.
ALPHA_8: A has 8 bits, A total of 1 byte, only transparency, no color.RGB_565: R contains 5 bits, G contains 6 bits, and B contains 5 bits, occupying 2 bytes in total.ARGB_4444: Each channel occupies four bits and two bytes. Poor display effect.ARGB_8888: Each channel occupies 8 bits and a total of 4 bytes.RGBA_F16: Each channel occupies 16 bits, occupying 8 bytes in total.
1.2. Memory usage
Images are stored in multiple ways, webP, PNG, JPG, etc., and they are compressed according to their own compression rules.
When an image needs to be loaded into memory, each pixel is loaded into memory, and the same pixel is not compressed or replaced.
So a simple formula is that the memory footprint of a Bitmap is equal to the Bitmap width * Bitmap height * bytes per pixel.
Bitmap, as we all know, has a problem with the OOM. The focus of Bitmap optimization is the memory problem. It can be seen from the formula that optimization can only be carried out in two aspects: width and height, and bytes occupied per unit pixel.
The number of bytes per pixel can only be adjusted by changing bitmap. config to change the pixel storage format. For example, if there is no ALPHA channel, changing from ARGB_8888 to RGB_565 will directly double the amount of space per pixel and the Bitmap will also directly double the amount of memory. It looks like a very happy result.
Although many places say that changing ARGB_8888 to RGB_565 does not have much effect on the display, the calculation shows that ARGB_8888 has 2^8 = 256 per channel, while RGB_565 only has 2^5 = 32. That’s a huge difference. This means that the quality of the image varies greatly, which is not recommended in most scenarios.
Since the number of bytes per unit is not recommended, you have to work on size.
Select bitmap. Options from bitmap. Options from bitmap. Options from bitmap. Options from bitmap. Options from bitmap. Options from bitmap. Options
inJustDecodeBounds: If settrueOnly the queryBitmap, but does not load the corresponding pixel data, used to obtain the image data, such as length, width and so on.inSampleSize: Sampling ratio, such as set to 4, will read the original image 4 * 4 pixel blocks into 1 * 1 pixel blocks. The exponent must be set to 2, otherwise round down.
1.3. Use sampling scale optimization
When it comes to displaying an image, the display area is limited, the DPI is limited, and so is what can be displayed. The pixel block of the picture is infinite, which is the first step of optimization. The resolution of the upper limit of the loading screen and the resolution of the original picture will not change the display effect. To put it more directly, a physical pixel block can only display the data of one image pixel block at most, so when loading an image, you only need to make the size larger than the physical pixel block.
MinimumDPi is the minimumDPi for image display to control the minimum loading density.
With the addition of DPI, the calculation of pixel px will be converted to the calculation of DP, and PX = DP * dPI, so the number of target pixels reqWidth/minimumDPi = vWidth * averageDpi, the ratio of the original size and target pixels is the sampling ratio.
val vWidth: Int = 0 // view width height,px
val vHeight: Int = 0
val sWidth: Int = 0 // Image width and height
val sHeight: Int = 0
val minimumDPi = 320 // You can set a minimum Dpi
private fun calculateInSampleSize(a): Int {
val metrics: DisplayMetrics = getResources().getDisplayMetrics()
val averageDpi = (metrics.xdpi + metrics.ydpi) / 2 / / screen dpi
/ / Dpi
val scaleDpi = 1f * minimumDPi / averageDpi
// The number of target pixels to display
val reqWidth = (vWidth * scaleDpi).toInt()
val reqHeight = (vHeight * scaleDpi).toInt()
var inSampleSize = 1
if (sHeight > reqHeight || sWidth > reqWidth) {
// The ratio of the number of pixels in the original image to the number of target pixels is the sampling ratio
val heightRatio = (1f * sHeight / reqHeight).roundToInt()
val widthRatio = (1f * sWidth / reqWidth).roundToInt()
// Take the smaller value, which corresponds to the display of FIT_CENTER.
inSampleSize = if (heightRatio < widthRatio) heightRatio else widthRatio
}
// Ensure the index of 2
var power = 1
while (power * 2 < inSampleSize) {
power *= 2
}
return if (power > 1) power / 2 else power
}
Copy the code2. Picture gesture operation
The method of sampling scale optimization was described earlier, dealing with a scenario where the image is only displayed in the control. In order to continue to analyze the picture in the control can zoom in and out and flat, first introduced the implementation of gesture operation.
2.1 coordinate system
In the view coordinate system, the origin is the upper left corner, the horizontal axis is X, the vertical axis is Y, and the unit length is pixels.
In the same way, set up an image coordinate system, starting with the upper left corner as the origin, the horizontal X axis, the vertical Y axis, the unit length is the original image pixel, the original image pixel. Again, it is important to say that the coordinate system is fixed only when the original image pixels are used, not necessarily after sampling.
2.2. Coordinate system deviation
In the process of zooming in, zooming in, and zooming out, the relationship between the two coordinate systems is actually changed.
Connect the two coordinate systems by defining two parameters to represent the scale and offset value.
scale: Zoom ratio. The ratio of the number of pixels in a view to the number of pixels in the original image in the same area.translate: Indicates the offset. The position of the image origin in the view coordinate system.
Using the defined scale and Translate, it is easy to convert the two coordinate systems to each other.
For example, if there is a point whose coordinate is vPoint in the view coordinate system and sPoint in the picture coordinate system, the relationship between the two should be:
vPoint = sPoint * scale + translate
Copy the codeThis completely confirms the position of a point in the view and image.
2.3. Parameter initialization
By default, images are displayed in FIT_CENTER, which is scaled up/down to the width of the View and centered. So at initialization:
- Enlarge/shrink the image proportionally to the width of the View, the smaller of the length or width
viewIf the width and height are the same, then the zoom ratio is the smaller value of the control size compared to the original image size. - The offset is half the difference between the control size and the image display size if the image is centered and aligned on one side.
fun initScaleAndTranslate(a){
scale = Math.min(vWidth / sWidth, vHeight / sHeight)
translate.x = (vWidth - scale * sWidth) / 2
translate.y = (vHeight - scale * sHeight) / 2
}
Copy the code2.4. Gesture processing
Then the relationship between gesture operation and zoom ratio and offset. Only the treatment of two fingers is introduced here.
In order for the gesture to follow the hand, the principle of processing is only one, the distance between the two fingers and the center point is fixed in the picture coordinate system.
In the view coordinate system, the distance D between two fingers and the center point PointCenter of two fingers correspond to the change of scale and Tanslation.
The distance s of finger is fixed in the picture coordinate system, and S * scale = D. The value of scale should be:
s1 = s2
d1 / scale1 = d2 / scale2
scale2 = scale1 * (d2 / d1)
Copy the codeThe finger is fixed at the center point of the picture coordinate system sPointCenter, vPoint = sPoint * scale + translation, translation should be:
sPointCenter1 = sPointCenter2
(vPointCenter1 - translation1) / scale1 = (vPointCenter2 - translation2) / scale2
vtranslation2 = vPointCenter2 - (vPointCenter1 - translation1) * (scale2 / scale1)
vtranslation2 = vPointCenter2 - (vPointCenter1 - translation1) * (d2 / d1)
Copy the codeHere is the concrete code implementation:
var scaleStart: Float = 0f
var vDistStart = 0f
var leftStart = 0f
var topStart = 0f
fun onTouchEvent(event: MotionEvent) {
if (event.pointerCount < 2) return
when (event.action) {
MotionEvent.ACTION_POINTER_2_DOWN -> {
// The absolute distance between the initial touch points of two fingers
vDistStart = distance(event.getX(0), event.getX(1), event.getY(0), event.getY(1))
scaleStart = scale
// The center coordinates of the initial contact points of the two fingers
val centerStartX = (event.getX(0) + event.getX(1)) / 2
val centerStartY = (event.getY(0) + event.getY(1)) / 2
// The median value calculated by the formula
leftStart = centerStartX - translate.x
topStart = centerStartY - translate.y
}
MotionEvent.ACTION_MOVE -> {
// The absolute distance between two fingers
val vDistEnd: Float = distance(event.getX(0), event.getX(1), event.getY(0), event.getY(1))
// The coordinates of the two fingers touching the center point
val vCenterEndX = (event.getX(0) + event.getX(1)) / 2
val vCenterEndY = (event.getY(0) + event.getY(1)) / 2
// Adjust the scale
scale = scaleStart * (vDistEnd / vDistStart)
// Adjust the offset value
val leftNow = leftStart * (vDistEnd / vDistStart)
val topNow = topStart * (vDistEnd / vDistStart)
translate.x = vCenterEndX - leftNow
translate.y = vCenterEndY - topNow
}
...
}
}
private fun distance(x0: Float, x1: Float, y0: Float, y1: Float): Float {
val x = x0 - x1
val y = y0 - y1
return sqrt(x * x + y * y)
}
Copy the code2.5. New sampling ratio calculation method
Scale defines the ratio of pixels in a view of the same area to pixels in the original image, which is the ratio of PX.
And what we derive from the sampling scale is the ratio of dp to Dpi, which is the inverse of scale times Dpi.
So the previous method of calculating the sampling ratio can be greatly simplified here.
val minimumDPi = 320 // Different minimum Dpi can be set
private fun calculateInSampleSize(a): Int {
val metrics: DisplayMetrics = getResources().getDisplayMetrics()
val averageDpi = (metrics.xdpi + metrics.ydpi) / 2 / / screen dpi
/ / Dpi
val scaleDpi = 1f * minimumDPi / averageDpi
val inSampleSize = (1f / scale / scaleDpi).roundToInt()
// Ensure the index of 2
var power = 1
while (power * 2 < inSampleSize) {
power *= 2
}
return if (power > 1) power / 2 else power
}
Copy the code3. Picture loading and display
In the process of interactive amplification, scale changes all the time, so the required sampling ratio may also change.
In this process, if the image is completely loaded with different sampling ratios each time, it will result in multiple resolutions of an image being loaded into memory, even larger than the size of the image being fully loaded.
The other change in sampling ratio is certainly accompanied by the enlargement of the image, at which time most of the image may not be displayed on the screen, which is a lot of unused memory usage.
3.1. BitmapRegionDecoder
BitmapRegionDecoder is a solution that can only load images of a given region.
BitmapRegionDecoder is relatively simple to use, creating a BitmapRegionDecoder object using newInstance(). The Boolean parameter at construction time indicates whether the stream being created is strongly referenced (false means strongly referenced).
val filePath = ""
val decoder = BitmapRegionDecoder.newInstance("imgPath".false)
Copy the codeOnce the object is created, the method decodeRegion() can be called to load the bitmap of the corresponding region, with parameters representing the loaded region and configuration.
val rect = Rect()
val options = BitmapFactory.Options().also {
it.inSampleSize = calculateInSampleSize()
}
val bitmap = decoder.decodeRegion(rect, options)
Copy the code3.2 block
With BitmapRegionDecoder, and can not be directly used to load, image operation process, will be very frequent changes in the display area, direct loading consumption is inestimable.
A better solution is to block load.
The basic principle of chunk loading is to slice up an image into small chunks and wait until that chunk enters the display area to load it.
Place the highest sampling ratio fully loaded as the background. After the image is enlarged to a new sampling ratio, the current image that needs to be displayed is quickly loaded in for rendering, which can well balance the problem between resolution and memory.
To facilitate block management, create a block object.
Each object maintains its own location information, as well as the required loading data and loading state.
After the partition, each piece is loaded independently, and loading the Bitmap is also a time-consuming operation. The asynchronous implementation of the Bitmap is loaded. This will trigger a redraw when the individual block is loaded.
class Tile(
sampleSize: Int.val sRect: Rect, // Load area
val decoder: BitmapRegionDecoder, / / loaders
val decoderLock: ReadWriteLock / / lock
) {
var bitmap: Bitmap? = null
var loading = false
var visible = false
private val options = BitmapFactory.Options()
init {
options.inSampleSize = sampleSize
}
/ / load the bitmap
suspend fun startLoadBitmap(a) {
if(loading || ! visible)return
loading = true
bitmap = withContext(Dispatchers.IO) {
tryLoadBitmap()
}
loading = false
}
private fun tryLoadBitmap(a): Bitmap? {
decoderLock.readLock().lock()
try {
return decoder.decodeRegion(sRect, options)
} catch (e: Exception) {
}
finally {
decoderLock.readLock().unlock()
}
return null}}Copy the codeThe load is triggered in the View and the coroutine is started using the lifecycleScope, ignoring the lifecycle processing.
Because of the blocking in tile.startloadBitmap (), the invalidate() call will not trigger the redraw until the bitmap is loaded.
fun loadBitmap(tile: Tile){ findViewTreeLifecycleOwner()? .lifecycleScope? .launch { tile.startLoadBitmap() invalidate() } }Copy the code3.3. Chunking logic
The triggering of partitioning should be done after initialization, get the highest sampling ratio, all possible sampling ratio of the whole good block.
For the highest sampling ratio is not divided into blocks, need to load in all, as a whole background display.
For other sampling ratios, it is necessary to divide them into blocks, and the rules of the block need not be too strict, as long as the size of the pixel block formed by the segmentation is close to the size of the control.
The size of the segmented block used here is not greater than 1.25 times the minimum number of segmented controls.
val tileMap = LinkedHashMap<Int, List<Tile>>() // For all slice objects, the key value is SampleSize and the insertion order is maintained
val decoder = BitmapRegionDecoder.newInstance("filePath".false) / / loaders
val decoderLock: ReadWriteLock = ReentrantReadWriteLock(true) / / lock
var maxSampleSize: Int = 32 // The sample rate at the minimum scale is the highest sample rate
fun initTiles(a) {
tileMap.clear()
maxSampleSize = calculateInSampleSize() // This is the maximum sampling rate at initialization
// First create a slice with the highest sampling rate and trigger the load. This slice is directly the size of the image as the background
val tile = Tile(
sampleSize = maxSampleSize,
sRect = Rect(0.0, sWidth, sHeight),
decoder = decoder,
decoderLock = decoderLock
)
tile.visible = true
loadBitmap(tile) // Trigger loading
tileMap[maxSampleSize] = listOf(tile)
// Create all slices recursively from the highest sampling rate /2 (sampling rate can only be 2 exponents)
var sampleSize = maxSampleSize / 2
// The number of slices in both directions
var xTiles = 1
var yTiles = 1
while (sampleSize > 1) {
// The original size of the slice
var sTileWidth: Int = sWidth / xTiles
var sTileHeight: Int = sHeight / yTiles
// The loading size of the slice
var subTileWidth = sTileWidth / sampleSize
var subTileHeight = sTileHeight / sampleSize
// Add slices until the size of each slice is no greater than the control size *1.25
while (subTileWidth > vWidth * 1.25) {
xTiles += 1
sTileWidth = sWidth / xTiles
subTileWidth = sTileWidth / sampleSize
}
while (subTileHeight > vHeight * 1.25) {
yTiles += 1
sTileHeight = sHeight / yTiles
subTileHeight = sTileHeight / sampleSize
}
/ / create a tile
val tileGrid = mutableListOf<Tile>()
for (x in 0 until xTiles) {
for (y in 0 until yTiles) {
tileGrid.add(
Tile(
sampleSize = sampleSize,
sRect = Rect(
x * sTileWidth, // Calculate the position in the image coordinate system according to the size and position of the slice
y * sTileHeight,
(x + 1) * sTileWidth,
(y + 1) * sTileHeight
),
decoder = decoder,
decoderLock = decoderLock
)
)
}
}
tileMap[sampleSize] = tileGrid
sampleSize /= 2}}Copy the code3.4. Status refresh
Once the chunks are sorted, there needs to be a logic that triggers loading and recycling, the principle being that it now triggers loading when it is in the display range and recycling when it is out of the display range.
The area of the control is mapped to the coordinate system of the picture, and then the position information of each maintenance is judged, and the overlap is loaded to display, and the non-overlap is triggered to recycle.
fun refreshTiles(a) {
val sampleSize = calculateInSampleSize() // Current sampling rate
// Map the control to the coordinate system of the image
val vRect = RectF()
vRect.left = (0 - translate.x) / scale
vRect.right = (vWidth - translate.x) / scale
vRect.top = (0 - translate.y) / scale
vRect.bottom = (vHeight - translate.y) / scale
tileMap.values.flatten().forEach { tile ->
if (tile.sampleSize == maxSampleSize) {
// Make sure the background bitmap is always there
tile.visible = true
if(! tile.loading && tile.bitmap ==null) {
loadBitmap(tile)
}
} else {
// The sampling ratio is equal and the loading is visible, otherwise the collection is triggered
if (tile.sampleSize == sampleSize && isTileVisible(tile.sRect, vRect)) {
tile.visible = true
if(! tile.loading && tile.bitmap ==null) {
loadBitmap(tile)
}
} else {
tile.visible = falsetile.bitmap? .recycle() tile.bitmap =null}}}}// Determine whether the two regions overlap
fun isTileVisible(sRect: Rect, vRect: RectF): Boolean {
return! (vRect.left > sRect.right || vRect.right < sRect.left || vRect.top > sRect.bottom || vRect.bottom < sRect.top) }Copy the code3.5. Draw
We’ve got everything up front, now we just need one, to draw these slices onto the screen.
Before drawing, check whether all tiles to display are loaded. If the load is complete, you only need to draw this part. If the load is not complete, you need to draw the background.
TileMap is a LinkedHashMap, which loops to ensure that it is rendered from the bottom up in the order of insertion, with lower resolution as you go down. That is, a low resolution background painting.
/ / to draw
override fun onDraw(canvas: Canvas?). {
if (tileMap.isEmpty()) {
return
}
val sampleSize = min(maxSampleSize, calculateInSampleSize())
// Check that all tiles to display are loaded
var hasMissingTiles = falsetileMap[sampleSize]? .forEach { tile ->if (tile.visible && (tile.loading || tile.bitmap == null)) {
hasMissingTiles = true}}// If the tile is loaded, just draw that part
// If the load is not complete, the tile will be attempted to draw
if (hasMissingTiles) {
tileMap.values.flatten()
} else{ tileMap[sampleSize] }? .filter { ! it.loading && it.bitmap ! =null}? .forEach { tile -> resetMatrix(tile) canvas? .drawBitmap(tile.bitmap ? :return, mMatrix, bitmapPaint)
}
}
val vRect = Rect()
val mMatrix = Matrix()
val bitmapPaint = Paint()
val srcArray = FloatArray(8)
val matrixArray = FloatArray(8)
// Calculate the matrix, map the image to the display area
fun resetMatrix(tile: Tile) {
mMatrix.reset()
valbitmap = tile.bitmap ? :return
srcArray[0] = 0f
srcArray[1] = 0f
srcArray[2] = bitmap.width.toFloat()
srcArray[3] = 0f
srcArray[4] = bitmap.width.toFloat()
srcArray[5] = bitmap.height.toFloat()
srcArray[6] = 0f
srcArray[7] = bitmap.height.toFloat()
sRectToVRect(tile.sRect, vRect)
matrixArray[0] = vRect.left.toFloat()
matrixArray[1] = vRect.top.toFloat()
matrixArray[2] = vRect.right.toFloat()
matrixArray[3] = vRect.top.toFloat()
matrixArray[4] = vRect.right.toFloat()
matrixArray[5] = vRect.bottom.toFloat()
matrixArray[6] = vRect.left.toFloat()
matrixArray[7] = vRect.bottom.toFloat()
mMatrix.setPolyToPoly(srcArray, 0, matrixArray, 0.4)}// Image coordinates map to control coordinates
fun sRectToVRect(sRect: Rect, vRect: Rect) {
vRect.set(
((sRect.left * scale) + translate.x).toInt(),
((sRect.top * scale) + translate.y).toInt(),
((sRect.right * scale) + translate.x).toInt(),
((sRect.bottom * scale) + translate.y).toInt()
)
}
Copy the codeWrite in the last
Here basically introduces the main process of a larger finished loading process, no processing for various boundary conditions, the article appears prudent use of code, the article reference SubsamplingScaleImageView, if there is a need to use it directly.
If you think the article is good, give it a thumbs up!
reference
Subsampling Scale Image View
A comprehensive summary of Bitmaps for Android development
Android Bitmap Optimization: Everything you need to know about Bitmaps
Android Bitmaps load efficiently, those little things you need to know