Image segmentation

To understand the watershed algorithm, we need to understand what is image segmentation.

In the process of image processing, it is often necessary to segment or extract foreground objects from images as target images. For example, in video surveillance, what is observed is the video content in the fixed background, but we are not interested in the background itself, but in the vehicles, pedestrians or other objects in the background. We want to pull these objects out of the video and ignore the video content that has no objects in the background.

Watershed algorithm

Image segmentation is a very important operation in image processing. The watershed algorithm can effectively compare the image to the terrain surface in geography, which is very useful for image segmentation.

Below, the blogger makes a brief introduction to the related content of the watershed algorithm. (For details, see Gonzales’s Digital Image Processing.)

Any grayscale image can be regarded as a geographical terrain surface. The region with higher grayscale value can be regarded as a mountain, while the region with lower grayscale value can be regarded as a valley.

If we fill each valley with a different color of water. So as the water level rises, the water from the different valleys comes together. In the process, in order to prevent the water from meeting in different valleys, we need to build DAMS where the water might meet. The process divides the image into two distinct sets: catchment basins and watershed lines. The dam we build is the watershed line, the segmentation of the original image. This is how the watershed algorithm works.

However, there is noise in the general image, and when the watershed algorithm is used, the result of excessive segmentation is often obtained. In order to improve the effect of image segmentation, an improved watershed algorithm based on mask is proposed. The improved watershed algorithm allows the user to mark out what it considers to be the same segmented region. In this way, the watershed algorithm will process the marked part into the same segmentation region when processing.

If you don’t know much about this theory, you can use the “Delete background” function in PowerPoint to observe and understand.

WaterShed function

In OpenCV, the watershed algorithm can be implemented using the function cv2.watershed() function. However, the specific implementation process also needs to use morphology function, distanceTransform function cv2.distancetransform (), cv2.connectedcomponents () to complete the image segmentation.

Morphological segmentation

Before using watershed algorithm, we need to do simple morphological processing on the image. In general, we use the open operation in morphology, because the operation is the operation of corrosion before expansion, which can remove the noise in the image.

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread("36.jpg")
k=np.ones((5.5),dtype=np.uint8)
e=cv2.erode(img,k)
result=cv2.subtract(img,e)

plt.subplot(131)
plt.imshow(img, cmap="gray")
plt.axis('off')

plt.subplot(132)
plt.imshow(e, cmap="gray")
plt.axis('off')

plt.subplot(133)
plt.imshow(result, cmap="gray")
plt.axis('off')
plt.show()
Copy the code

Recall that our previous open operation is cv2.erode(), where we first open the operation to remove the noise. Subtract (cv2.subtract()) to obtain the image boundary. After running, the effect is as follows:

DistanceTransform function

When the subgraphs in the image are not connected, the foreground object can be directly determined by morphological corrosion operation, but if the subgraphs in the image are connected together, it is difficult to determine the foreground object. At this time, you need to use the transform function cv2.distanceTransform() to easily extract the foreground object.

Cv2.distancetransform () reflects the distance relationship between each pixel and the background (pixel with value 0). Usually:

  1. If the center of the foreground object is further away from the pixel with a value of 0, a larger value will be obtained.
  2. If the foreground object’s edge is closer to a pixel with a value of 0, a smaller value will be obtained.

Next, let’s use this function to determine the foreground of an image and observe the effect.

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread("36.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 0.255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
k = np.ones((5.5), dtype=np.uint8)
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, k, iterations=2)
distTransform = cv2.distanceTransform(opening, cv2.DIST_L2, 5)
ret, fore = cv2.threshold(distTransform, 0.7 * distTransform.max(), 255.0)

plt.subplot(131)
plt.imshow(img, cmap="gray")
plt.axis('off')

plt.subplot(132)
plt.imshow(distTransform, cmap="gray")
plt.axis('off')

plt.subplot(133)
plt.imshow(fore, cmap="gray")
plt.axis('off')
plt.show()
Copy the code

Here, we use cv2.Morphologyex function for open operation and cv2.distanceTransform to get the distance image. Finally, we use CV2.threshold for threshold processing of the distance image to determine the foreground. After running, the effect is as follows:

Identification of unknown areas

Through the distance function, we obtain the “center” of the image, that is, “determine the foreground”. For the convenience of subsequent explanation, we will define the foreground as F.

With determined foreground F and determined background B in the image, the remaining region is the unknown region UN. This part of the region is the watershed algorithm to further clarify the region.

For an image 0, the unknown region UN can be obtained through the following relationship:

Unknown area UN= Image 0- Determine background B- Determine foreground F

By the transformation of the above formula, it can be obtained:

Unknown region UN= (Image 0- determine background B) – Determine foreground F

Where (image 0- determine background B) is the subtraction operation we started and obtained through morphological expansion. Just add 4 lines to the above code and change the code content displayed:

bg=cv2.dilate(opening,k,iterations=3)
fore=np.uint8(fore)
un=cv2.subtract(bg,fore)

plt.subplot(221)
plt.imshow(img, cmap="gray")
plt.axis('off')

plt.subplot(222)
plt.imshow(bg, cmap="gray")
plt.axis('off')

plt.subplot(223)
plt.imshow(fore, cmap="gray")
plt.axis('off')

plt.subplot(224)
plt.imshow(un, cmap="gray")
plt.axis('off')
plt.show()
Copy the code

After running, the effect is as follows:

Top left is the original image

On the upper right is the image BG obtained after expansion of the original image, and its background image is the determined background B. The foreground image is “Original image 0- Determine background B”

The lower left is fore

At the lower right is the unknown region image UN

ConnectedComponents function

After the definite prospect is made clear, it can be marked for the definite prospect. In OpenCV, it provides the cv2.ConnectedComponents() function for annotation.

This function marks the background as 0 and other objects as positive integers starting at 1. It has only one parameter 8-bit single-channel image to be annotated.

Retval returns two values: retval returns the number of labels, and Labels returns the result image.

Next, let’s use this function for annotation. The code is as follows (again, change the code below bg above) :

bg = cv2.dilate(opening, k, iterations=3)
fore = np.uint8(fore)
ret, markets = cv2.connectedComponents(fore)
unknown=cv2.subtract(bg,fore)
markets=markets+1
markets[unknown==255] =0

plt.subplot(131)
plt.imshow(img, cmap="gray")
plt.axis('off')

plt.subplot(132)
plt.imshow(fore, cmap="gray")
plt.axis('off')

plt.subplot(133)
plt.imshow(markets, cmap="gray")
plt.axis('off')
plt.show()
Copy the code

Modify the fore = Np.Uint8 (fore) code and modify the output. After running, we get the original image, fore, the center point image, and Markets, the annotated result image. The effect is as follows:

Practical watershed algorithm

After the previous introduction, we understand the basic steps of image segmentation using watershed algorithm:

  1. The original image 0 was denoised by morphological operation
  2. Obtain Determine Background B by corrosion operation. Note that you just get “Raw image – Determine background”
  3. The distance transformation function is used to calculate the original image, and the threshold value is processed to get “Determine foreground F”.
  4. Calculate the unknown region UN (UN= 0-b-f)
  5. The function cv2.ConnectedComponents () was used to annotate the original image 0
  6. Fix the annotation results for function cv2.ConnectedComponents ()
  7. Use watershed function to complete image segmentation

The complete code is as follows:

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread("36.jpg")
plt.subplot(121)
plt.imshow(img, cmap="gray")
plt.axis('off')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 0.255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
k = np.ones((5.5), dtype=np.uint8)
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, k, iterations=2)
distTransform = cv2.distanceTransform(opening, cv2.DIST_L2, 5)
ret, fore = cv2.threshold(distTransform, 0.2 * distTransform.max(), 255.0)
bg = cv2.dilate(opening, k, iterations=3)
fore = np.uint8(fore)
ret, markets = cv2.connectedComponents(fore)
unknown = cv2.subtract(bg, fore)
markets = markets + 1
markets[unknown == 255] = 0
markets = cv2.watershed(img, markets)
img[markets == -1] = [255.0.0]

plt.subplot(122)
plt.imshow(img, cmap="gray")
plt.axis('off')
plt.show()
Copy the code

After running, we can get the segmtioned image:

Of course, the parameters can be adjusted, and you can see that the approximate coin is fully divided.