“This is my 32nd day of participating in the First Challenge 2022. For more details: First Challenge 2022.”

I. PaddleHub cat matting with one key based on Oxford-IIIT Pet data set

1. Main work

• Cat data set was extracted from Oxford-IIIT Pet data set
• Re-create the label with the background set to 0 and the image set to 1
• At the end of ITER 2100, mIoU =0.7874, there is still room for rise, but it takes a long time and no more training
• Deploy the exported static model through paddleHub

Gz for model files and catseg_mobile.zip for paddlehub deployment.

Finally you can be perfect matting, you can make a cat id photo.

2. PaddleSeg profile

PaddleSeg is an end-to-end image segmentation development kit based on PaddlePaddle, which covers a large number of high quality segmentation models in different directions, including high precision and lightweight. Through the modular design, it provides two application methods of configuration-driven and API call, which helps developers to complete the whole process of image segmentation application from training to deployment more conveniently.

features

• High-precision model: The high-precision backbone network is obtained based on the semi-supervised label knowledge distillation scheme (SSLD) training developed by Baidu itself, and 50+ high-quality pre-training model is provided in combination with cutting-edge segmentation technology. The effect is better than other open source implementations.
• Modular design: support 15+ mainstream segmentation network, combined with modular design of different components such as data enhancement strategy, backbone network, loss function, developers can assemble a variety of training configurations based on actual application scenarios, to meet different performance and accuracy requirements.
• High performance: support multi-process asynchronous I/O, multi-card parallel training, evaluation and other acceleration strategies, combined with the video memory optimization function of the flying-blade core framework, can greatly reduce the training cost of segmentation model, so that developers can complete image segmentation training at a lower cost and more efficiently.

! git clone https://gitee.com/paddlepaddle/PaddleSeg.git  --depth=1
Copy the code

Cloning into 'PaddleSeg'... remote: Enumerating objects: 1589, done.[K remote: Counting objects: 100% (1589/1589), done.[K remote: Compressing objects: 100% (1354/1354), done.[K remote: Total 1589 (delta 309), reused 1117 (delta 142), pack-reused 0[K Receiving objects: 100% (1589/1589) and 88.49 MiB | 5.57 MiB/s, done. Resolving deltas: 100% (309/309), done. Checking connectivity... done.Copy the code

3. Data set making

Need to manually delete the dataset/annotations/list. TXT file header, easy to read pandas, such as trouble, can be used directly has been produced good data set 2, cat data set.

Unzip the dataset! mkdir dataset ! tar -xvf data/data50154/images.tar.gz -C dataset/ ! tar -xvf data/data50154/annotations.tar.gz -C dataset/Copy the code

# Check the list file! head -n10 dataset/annotations/list.txt
Copy the code

#Image CLASS-ID SPECIES BREED ID
#ID: 1:37 Class ids
#SPECIES: 1:Cat 2:Dog
#BREED ID: 1-25:Cat 1:12:Dog
#All images with 1st letter as captial are cat images
#images with small first letter are dog images
Abyssinian_100 1 1 1
Abyssinian_101 1 1 1
Abyssinian_102 1 1 1
Abyssinian_103 1 1 1
Copy the code

Delete the first 6 lines of the file to be read by pandas! sed -i'1, 6 d' dataset/annotations/list.txt
Copy the code

! head dataset/annotations/list.txt
Copy the code

Abyssinian_100 1 1 1
Abyssinian_101 1 1 1
Abyssinian_102 1 1 1
Abyssinian_103 1 1 1
Abyssinian_104 1 1 1
Abyssinian_105 1 1 1
Abyssinian_106 1 1 1
Abyssinian_107 1 1 1
Abyssinian_108 1 1 1
Abyssinian_109 1 1 1
Copy the code

import pandas as pd
import shutil
import os


# Image CLASS-ID SPECIES BREED ID
# ID: 1:37 Class ids
# SPECIES: 1:Cat 2:Dog
# BREED ID: 1-25:Cat 1:12:Dog
# All images with 1st letter as captial are cat images
# images with small first letter are dog images
# ._Abyssinian_100.png

def copyfile(animal, filename) :
    # image \ label list
    file_list = []
    image_file = filename + '.jpg'
    label_file = filename + '.png'

    if os.path.exists(os.path.join('dataset/images', image_file)):
        shutil.copy(os.path.join('dataset/images', image_file), os.path.join(f'{animal}/images', image_file))
        shutil.copy(os.path.join('dataset/annotations/trimaps', label_file),
                    os.path.join(f'{animal}/labels', label_file))
        temp = os.path.join('images/', image_file) + ' ' + os.path.join('labels/',label_file) + '\n'
        file_list.append(temp)
    with open(os.path.join(animal, animal + '.txt'), 'a') as f:
        f.writelines(file_list)


if __name__ == "__main__":

    data = pd.read_csv('dataset/annotations/list.txt', header=None, sep=' ')
    data.head()

    cat = data[data[2] = =1]
    dog = data[data[2] = =2]

    for item in cat[0]:
        copyfile('cat', item)

    for item in dog[0]:
        copyfile('dog', item)

Copy the code

Delete unnecessary data! rm dataset/ -rfCopy the code

4. Train custom data sets

4.1 File Structure

├ ─ ─ the TXT ├ ─ ─ images │ ├ ─ ─ Abyssinian_100. JPG │ ├ ─ ─ Abyssinian_101. JPG │ ├ ─ ─... ├ ─ ─ labels │ ├ ─ ─ Abyssinian_100. PNG │ ├ ─ ─ Abyssinian_101. PNG │ ├ ─ ─...Copy the code

4.2 List Contents:

images/Abyssinian_1.jpg labels/Abyssinian_1.png
images/Abyssinian_10.jpg labels/Abyssinian_10.png
images/Abyssinian_100.jpg labels/Abyssinian_100.png
...
Copy the code

4.3. Data viewing

%cd ~
from PIL import Image

img=Image.open('cat/images/Abyssinian_123.jpg')
print(img)
img
Copy the code

/home/aistudio
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=500x333 at 0x7F203C05FBD0>
Copy the code

img=Image.open('cat/labels/Abyssinian_123.png')
print(img)
img
Copy the code

<PIL.PngImagePlugin.PngImageFile image mode=L size=500x333 at 0x7F203C0574D0>
Copy the code

5. Label processing

The tags are sorted from 0. Data for this project was extracted from Oxford-IIIT Pet www.robots.ox.ac.uk/~vgg/data/p… Pet data set, which is encoded from 1, so it needs to be re-encoded. The background is set to 0 and the image to 1.

Run this command once
import pandas as pd
import os
import matplotlib.pyplot as plt
import numpy as np
from PIL import Image

def re_label(filename) :
    img = plt.imread(filename) * 255.0
    img_label = np.zeros((img.shape[0], img.shape[1]), np.uint8)
    for i in range(img.shape[0) :for j in range(img.shape[1]):
            value = img[i, j]
            if value == 2:
                img_label[i, j] = 1
    label0 = Image.fromarray(np.uint8(img_label))
    label0.save( filename)

data=pd.read_csv("cat/cat.txt", header=None, sep=' ') 
for item in data[1]:
    re_label(os.path.join('cat', item))
print('Done! ')    
Copy the code

Done!Copy the code

2. Data set preprocessing

import os
from sklearn.model_selection import train_test_split
import pandas as pd


def break_data(target, rate=0.2) :
    origin_dataset = pd.read_csv("cat/cat.txt", header=None, sep=' ')  # add parameter
    train_data, test_data = train_test_split(origin_dataset, test_size=rate)
    train_data,eval_data=train_test_split(train_data, test_size=rate)
    train_filename = os.path.join(target, 'train.txt')
    test_filename = os.path.join(target, 'test.txt')
    eval_filename = os.path.join(target, 'eval.txt')

    train_data.to_csv(train_filename, index=False, sep=' ',  header=None)
    test_data.to_csv(test_filename, index=False, sep=' ', header=None)
    eval_data.to_csv(eval_filename, index=False, sep=' ', header=None)

    print('train_data:'.len(train_data))
    print('test_data:'.len(test_data))
    print('eval_data:'.len(eval_data))

if __name__ == '__main__':
    break_data(target='cat', rate=0.2)
Copy the code

train_data: 1516
test_data: 475
eval_data: 380
Copy the code

# check! head cat/train.txtCopy the code

images/Bengal_173.jpg labels/Bengal_173.png
images/Siamese_179.jpg labels/Siamese_179.png
images/British_Shorthair_201.jpg labels/British_Shorthair_201.png
images/Russian_Blue_60.jpg labels/Russian_Blue_60.png
images/British_Shorthair_93.jpg labels/British_Shorthair_93.png
images/British_Shorthair_26.jpg labels/British_Shorthair_26.png
images/British_Shorthair_209.jpg labels/British_Shorthair_209.png
images/British_Shorthair_101.jpg labels/British_Shorthair_101.png
images/British_Shorthair_269.jpg labels/British_Shorthair_269.png
images/Ragdoll_59.jpg labels/Ragdoll_59.png
Copy the code

Three, configuration,

The configuration is complete and no copy is needed
# !cp PaddleSeg/configs/quick_start/bisenet_optic_disc_512x512_1k.yml ~/bisenet_optic_disc_512x512_1k.yml
Copy the code

To modify bisenet_optic_disc_512x512_1k.yml, note the following:

• 1. Set the path of the data set
• 2. Num_classes setting, background does not count
• 3. Transforms Settings
• 4. Loss Settings

batch_size: 600 iters: 5000 train_dataset: type: Dataset dataset_root: /home/aistudio/cat/ train_path: /home/aistudio/cat/train.txt num_classes: 2 transforms: - type: ResizeStepScaling min_scale_factor: Max_scale_factor: 2.0 scale_step_size: 0.25 - type: RandomPaddingCrop crop_size: [224, 224] - type: Randomhorizontalflip-type: RandomDistort brightness_range: 0.4 Contrast_range: 0.4 saturation_range: 0.4 - type: Normalize mode: train val_dataset: type: Dataset dataset_root: /home/aistudio/cat/ val_path: /home/aistudio/cat/eval.txt num_classes: 2 transforms: - type: Normalize mode: val optimizer: type: sgd momentum: 0.9 weight_decay: 0.0005 lR_scheduler: type: PolynomialDecay Learning_rate: 0.05 end_LR: 0 power: 0.9 Loss: types: - type: CrossEntropyLoss coef: [1] model: type: FCN backbone: type: HRNet_W18_Small_V1 align_corners: False num_classes: 2 pretrained: NullCopy the code

Fourth, training

%cd ~/PaddleSeg/ ! python train.py --config .. /bisenet_optic_disc_512x512_1k.yml\ --do_eval \ --use_vdl \ --save_interval100 \
    --save_dir output
Copy the code

2021-11-13 19:30:52 [INFO] [TRAIN] epoch: 1105, ITER: 2210/5000, Loss: 0.1849, LR: 0.029586, batCH_cost: 8.8180, reader_cost: 7.73956, ips: 68.0427 ETA 06:50:02 2021-11-13 samples/SEC | 19:32:18 [INFO] [TRAIN] epoch: 1110, iter: 2220/5000, Loss: 0.1768, LR: 0.029490, BATCH_cost: 8.6004, reader_cost: 7.52235, IPS: 69.7641 ETA 06:38:29 2021-11-13 samples/SEC | 19:33:47 [INFO] [TRAIN] epoch: 1115, iter: 2230/5000, loss: 0.1791, lr: 0.029395, batCH_cost: 8.8851, reader_cost: 7.80702, ips: 67.5288 ETA 06:50:11 2021-11-13 samples/SEC | 19:35:14 [INFO] [TRAIN] epoch: 1120, iter: 2240/5000, loss: 0.1835, lr: 0.029299, batCH_cost: 8.6699, reader_cost: 7.59314, ips: 69.2053 ETA 06:38:48 2021-11-13 samples/SEC | 19:36:41 [INFO] [TRAIN] epoch: 1125, iter: 2250/5000, loss: 0.1815, lr: 0.029204, batCH_cost: 8.7713, reader_cost: 7.68169, ips: 68.4051 ETA 06:42:00 2021-11-13 samples/SEC | 19:38:08 [INFO] [TRAIN] epoch: 1130, iter: 2260/5000, loss: 0.1833, lr: 0.029108, batCH_cost: 8.7045, reader_cost: 7.62504, ips: 68.9299 ETA 06:37:30 2021-11-13 samples/SEC | 19:39:35 [INFO] [TRAIN] epoch: 1135, iter: 2270/5000, loss: 0.1741, lr: 0.029013, batCH_cost: 8.7032, reader_cost: 7.61708, ips: 68.9401 ETA 06:35:59 2021-11-13 samples/SEC | 19:41:03 [INFO] [TRAIN] epoch: 1140, iter: 2280/5000, loss: 0.1810, lr: 0.028917, batCH_cost: 8.8020, reader_cost: 7.72264, ips: 68.1664 ETA 06:39:01 2021-11-13 samples/SEC | 19:42:33 [INFO] [TRAIN] epoch: 1145, iter: 2290/5000, loss: 0.1799, lr: 0.028821 batCH_cost: 8.9336 reader_cost: 7.84692, ips: 67.1623 ETA 06:43:30 2021-11-13 samples/SEC | 19:44:02 [INFO] [TRAIN] epoch: 1150, iter: 2300/5000, loss: 0.1756, lr: 0.028726, batCH_cost: 8.9216, reader_cost: 7.84517, ips: 67.2524 ETA 06:41:28 2021-11-13 samples/SEC | 19:44:02 [INFO] Start evaluating (total_samples: 380, total_iters: 380)... 380/380 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 15 to 40 ms/s step - batch_cost: 0.0394 - reader cost: 0.001 2021-11-13 19:44:17 [INFO] [EVAL] #Images: 380 mIoU: 0.7640 Acc: 0.8681 Kappa: 2021-11-13 19:44:17 [INFO] [EVAL] Class IoU: [0.7378 0.7902] 2021-11-13 19:44:17 [INFO] [EVAL] Class Acc: [INFO] [EVAL] The model with The best Validation mIoU (0.7874) was saved at iter 2100.Copy the code

Five, the test

! python val.py \ --config /home/aistudio/bisenet_optic_disc_512x512_1k.yml\ --model_path output/best_model/model.pdparamsCopy the code

2021-11-13 19:48:13 [INFO] ---------------Config Information--------------- batch_size: 600 iters: 5000 loss: coef: -1 types: - type: CrossEntropyLoss LR_scheduler: end_LR: 0 Learning_rate: 0.05 Power: 0.9 Type: theoreialdecay model: backbone: align_corners: false type: HRNet_W18_Small_V1 num_classes: 2 pretrained: null type: FCN optimizer: momentum: 0.9 type: SGD weight_decay: 0.0005 train_dataset: dataset_root: /home/aistudio/cat/ mode: train num_classes: 2 train_path: / home/aistudio/cat/train. TXT transforms: - max_scale_factor: min_scale_factor 2.0:0.5 scale_step_size: 0.25 type: ResizeStepScaling - crop_size: -224-224 type: randompaddingcorp-type: RandomHorizontalFlip - brightness_range: 0.4 Contrast_range: 0.4 saturation_range: 0.4 type: RandomDistort - type: Normalize type: Dataset val_dataset: dataset_root: /home/aistudio/cat/ mode: val num_classes: 2 transforms: - type: Normalize type: Dataset val_path: / home/aistudio/cat/eval. TXT -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- W1113 19:48:13. 707370, 4265 Device_context. cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: Runtime API Version: 10.1 W1113 19:48:13.707428 4265 Device_context. cc:422] DEVICE: 0, cuDNN Version: 2021-11-1319:48:19 [INFO] Loading Pretrained Model from Output/Best_model/model.pdParams 2021-11-1319:48:19 [INFO] There are 363/363 variables loaded into FCN. 2021-11-13 19:48:19 [INFO] Loaded trained params of model successfully 2021-11-13 19:48:19 [INFO] Start evaluating (total_samples: 380, total_iters: 380)... / opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/paddle/fluid/dygraph/math_op_patch py: 239: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.int32, but right dtype is paddle.bool, the right dtype will convert to paddle.int32 format(lhs_dtype, rhs_dtype, lhs_dtype)) / opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/paddle/fluid/dygraph/math_op_patch py: 239: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.int64, but right dtype is paddle.bool, the right dtype will convert to paddle.int64 format(lhs_dtype, rhs_dtype, 380/380 [lhs_dtype)) = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 15 s 41 ms/step - batch_cost: 0.0405 - reader cost: 0.00 2021-11-13 19:48:35 [INFO] [EVAL] #Images: 380 mIoU: 0.7874 Acc: 0.8838 Kappa: 2021-11-13 19:48:35 [INFO] [EVAL] Class IoU: [0.7566 0.8181] 2021-11-13 19:48:35 [INFO] [EVAL] Class Acc: [0.8349 0.9211]Copy the code

380/380 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 15 s 41 ms/step - batch_cost: 0.0405 - reader cost: 0.00 2021-11-13 19:48:35 [INFO] [EVAL] #Images: 380 mIoU: 0.7874 Acc: 0.8838 Kappa: 2021-11-13 19:48:35 [INFO] [EVAL] Class IoU: [0.7566 0.8181] 2021-11-13 19:48:35 [INFO] [EVAL] Class Acc: [0.8349 0.9211]Copy the code

6. Export the static model

! python export.py \ --config /home/aistudio/bisenet_optic_disc_512x512_1k.yml\ --model_path output/best_model/model.pdparamsCopy the code

op_type, op_type, EXPRESSION_MAP[method_name])) / opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/paddle/fluid/the layers/math_op_patch py: 322: UserWarning: /tmp/tmp_l3u6xjv.py:58 The behavior of expression A + B has been unified with elementwise_add(X, Y, Axis =-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future. op_type, op_type, EXPRESSION_MAP[method_name])) 2021-11-03 01:01:11 [INFO] Model is saved in ./output.Copy the code

Seven, forecasting

deploy.yaml

Deploy:
  model: model.pdmodel
  params: model.pdiparams
  transforms:
  - type: Normalize
Copy the code

# installation paddleseg! pip install -e .Copy the code

# prediction%cd ~/PaddleSeg/ ! python deploy/python/infer.py --config output/deploy.yaml --image_path /home/aistudio/cat/images/Bombay_130.jpgCopy the code

/home/aistudio/PaddleSeg
Copy the code

# Print the original
from PIL import Image
img=Image.open('/home/aistudio/cat/images/Bombay_130.jpg')
img
Copy the code

Print the output graph with adjustable color
from PIL import Image
img=Image.open('/home/aistudio/PaddleSeg/output/Bombay_130.png')
img
Copy the code

Eighth, hub deployment

For hub deployment, see PaddleHub Module conversion

Hub deployed, you can matting via command line or Python. For the specific hub file, see the directory compression package catseg_mobile.zip

hub run catseg_mobile --input_path .\cat1.jpg
Copy the code

Copy the code