“This is the 11th day of my participation in the First Challenge 2022. For details: First Challenge 2022.”

The preface

How to prevent hard-trained models from leaking when deploying in C++? How should deep learning models be kept secret?

This article describes a simple model encryption method provided by PaddleX. Although there is a certain gap from absolute confidentiality, but after all, there is no problem.

This week is National Cybersecurity Awareness Week, so let’s take a look at how paddling can make deep learning models safer

Project introduction

This article introduces the implementation of yolov3_Darknet53 helmet detection migration learning based on PaddleX.

Helmet wearing detection is a typical scenario applied by computer vision in the field of industrial security. PaddleX is used for pp-yolo transfer learning and training, and three deployment modes including Python deployment, local C++ model encryption deployment and paddlehub-serving deployment are provided.

In this paper, the transfer learning performance of PP-YOLO and Yolov3_Darknet53 on the helmet detection dataset is also compared.

About the Project

For the improvement of the project, such as the deployment of encryption in other environments, I hope we can exchange views, introduce experience, and learn and progress together. Personal home page

Environment to prepare

Installation Tool library

! pip install ipywidgetsCopy the code

Extract data set

! mkdir MyDatasetCopy the code

! unzip data/data50329/HelmetDetection.zip -d ./MyDataset
Copy the code

Shard the data set

Option 1: Use the latest Develop branch of PaddleX (not recommended)

The implementation method has been described in the training and One-click Deployment (PaddleX, HubServing) of the helmet wearing detection model for the pre-project.

#! Paddlex --split_dataset --format VOC -- dataset_DIR MyDataset --val_value 0.2 --test_value 0.1
Copy the code

Dataset Split Done.[0m
[0mTrain samples: 3500[0m
[0mEval samples: 1000[0m
[0mTest samples: 500[0m
[0mSplit files saved in MyDataset[0m
[0m[0m[0m
Copy the code

Plan 2: Rewrite the data set splitting code by referring to the Voc_split.py of the Develop branch

The steps are as follows:

1. Install PaddleX from PIP
2. willvoc_split.pyFind the methods to import and run them in the Notebook
3. Change the voc_split.py file directory name for the split data set

# PIP install PaddleX! pip install paddlexCopy the code

# PaddleX/paddlex/tools/dataset_split/utils.py

# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os
import os.path as osp
from PIL import Image
import numpy as np
import json


class MyEncoder(json.JSONEncoder) :
    Modify the JSON file storage format
    def default(self, obj) :
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(MyEncoder, self).default(obj)


def list_files(dirname) :
    Args: dirName: directory path Args: dirName: directory path

    def filter_file(f) :
        if f.startswith('. ') :return True
        return False

    all_files = list()
    dirs = list(a)for f in os.listdir(dirname):
        if filter_file(f):
            continue
        if osp.isdir(osp.join(dirname, f)):
            dirs.append(f)
        else:
            all_files.append(f)
    for d in dirs:
        for f in os.listdir(osp.join(dirname, d)):
            if filter_file(f):
                continue
            if osp.isdir(osp.join(dirname, d, f)):
                continue
            all_files.append(osp.join(d, f))
    return all_files


def is_pic(filename) :
    Args: filename: file path ""
    suffixes = {'JPEG'.'jpeg'.'JPG'.'jpg'.'BMP'.'bmp'.'PNG'.'png'}
    suffix = filename.strip().split('. ')[-1]
    if suffix not in suffixes:
        return False
    return True


def replace_ext(filename, new_ext) :
    """ Replace file suffix Args: filename: file path new_ext: new suffix to be replaced """
    items = filename.split(".")
    items[-1] = new_ext
    new_filename = ".".join(items)
    return new_filename


def read_seg_ann(pngfile) :
    Args: pngFile: the path of a PNG image containing the annotated information.
    grt = np.asarray(Image.open(pngfile))
    labels = list(np.unique(grt))
    if 255 in labels:
        labels.remove(255)
    return labels

Copy the code

# PaddleX/paddlex/utils/logging.py

# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import time
import os
import sys
import colorama
from colorama import init
import paddlex

init(autoreset=True)
levels = {0: 'ERROR'.1: 'WARNING'.2: 'INFO'.3: 'DEBUG'}


def log(level=2, message="", use_color=False) :
    current_time = time.time()
    time_array = time.localtime(current_time)
    current_time = time.strftime("%Y-%m-%d %H:%M:%S", time_array)
    if paddlex.log_level >= level:
        if use_color:
            print("\033[1;31;40m{} [{}]\t{}\033[0m".format(
                current_time, levels[level], message).encode("utf-8").decode(
                    "latin1"))
        else:
            print("{} [{}]\t{}".format(current_time, levels[level], message)
                  .encode("utf-8").decode("latin1"))
        sys.stdout.flush()


def debug(message="", use_color=False) :
    log(level=3, message=message, use_color=use_color)


def info(message="", use_color=False) :
    log(level=2, message=message, use_color=use_color)


def warning(message="", use_color=True) :
    log(level=1, message=message, use_color=use_color)


def error(message="", use_color=True, exit=True) :
    log(level=0, message=message, use_color=use_color)
    if exit:
        sys.exit(-1)

Copy the code

The 2020-09-13 10:07:00, 072 - the INFO: The font search path ['/opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/matplotlib/MPL - data/fonts/the vera.ttf ', '/ opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/matplotlib/MPL - data/fonts/afm', '/ opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/matplotlib/MPL - data/fonts/pdfcorefonts'] in the 2020-09-13 s 10:07:00, 691 - the INFO: generated new fontManagerCopy the code

# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os.path as osp
import random
import xml.etree.ElementTree as ET


def split_voc_dataset(dataset_dir, val_percent, test_percent, save_dir) :
    # Note that the image directory and label directory names have been changed
    if not osp.exists(osp.join(dataset_dir, "images")):
        logging.error("\'images\' is not found in {}!".format(dataset_dir))
    if not osp.exists(osp.join(dataset_dir, "annotations")):
        logging.error("\'annotations\' is not found in {}!".format(
            dataset_dir))

    all_image_files = list_files(osp.join(dataset_dir, "images"))

    image_anno_list = list()
    label_list = list(a)for image_file in all_image_files:
        if not is_pic(image_file):
            continue
        anno_name = replace_ext(image_file, "xml")
        if osp.exists(osp.join(dataset_dir, "annotations", anno_name)):
            image_anno_list.append([image_file, anno_name])
            try:
                tree = ET.parse(
                    osp.join(dataset_dir, "annotations", anno_name))
            except:
                raise Exception("File {} is not a well-formed XML file, please check the annotation file".format(
                    osp.join(dataset_dir, "annotations", anno_name)))
            objs = tree.findall("object")
            for i, obj in enumerate(objs):
                cname = obj.find('name').text
                if not cname in label_list:
                    label_list.append(cname)
        else:
            logging.error("The annotation file {} doesn't exist!".format(
                anno_name))

    random.shuffle(image_anno_list)
    image_num = len(image_anno_list)
    val_num = int(image_num * val_percent)
    test_num = int(image_num * test_percent)
    train_num = image_num - val_num - test_num

    train_image_anno_list = image_anno_list[:train_num]
    val_image_anno_list = image_anno_list[train_num:train_num + val_num]
    test_image_anno_list = image_anno_list[train_num + val_num:]

    with open(
            osp.join(save_dir, 'train_list.txt'), mode='w',
            encoding='utf-8') as f:
        for x in train_image_anno_list:
            file = osp.join("images", x[0])
            label = osp.join("annotations", x[1])
            f.write('{} {}\n'.format(file, label))
    with open(
            osp.join(save_dir, 'val_list.txt'), mode='w',
            encoding='utf-8') as f:
        for x in val_image_anno_list:
            file = osp.join("images", x[0])
            label = osp.join("annotations", x[1])
            f.write('{} {}\n'.format(file, label))
    if len(test_image_anno_list):
        with open(
                osp.join(save_dir, 'test_list.txt'), mode='w',
                encoding='utf-8') as f:
            for x in test_image_anno_list:
                file = osp.join("images", x[0])
                label = osp.join("annotations", x[1])
                f.write('{} {}\n'.format(file, label))
    with open(
            osp.join(save_dir, 'labels.txt'), mode='w', encoding='utf-8') as f:
        for l in sorted(label_list):
            f.write('{}\n'.format(l))

    return train_num, val_num, test_num


if __name__ == "__main__":
    # Shard the data set
    split_voc_dataset('MyDataset'.0.2.0.1.'MyDataset')

Copy the code

Comparison of pp-YOLO and Yolov3_darknet53 training performance

The official document PP-YOLO training code is used directly here.

See output/ ppyOLO/vDL_log for VisualDL training process.

1. Decline Trend of Loss

2. Change of learning rate

3. Bbox_map changes on validation set

Start training

# Environment variable configuration, used to control whether to use the GPU
# documentation: https://paddlex.readthedocs.io/zh_CN/develop/appendix/parameters.html#gpu
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

from paddlex.det import transforms
import paddlex as pdx

Defining transforms for training and validation
# https://paddlex.readthedocs.io/zh_CN/develop/apis/transforms/det_transforms.html API
train_transforms = transforms.Compose([
    transforms.MixupImage(mixup_epoch=250), transforms.RandomDistort(),
    transforms.RandomExpand(), transforms.RandomCrop(), transforms.Resize(
        target_size=608, interp='RANDOM'), transforms.RandomHorizontalFlip(),
    transforms.Normalize()
])

eval_transforms = transforms.Compose([
    transforms.Resize(
        target_size=608, interp='CUBIC'), transforms.Normalize()
])

Define the data set for training and validation
# API description: https://paddlex.readthedocs.io/zh_CN/develop/apis/datasets.html#paddlex-datasets-vocdetection
train_dataset = pdx.datasets.VOCDetection(
    data_dir='MyDataset',
    file_list='MyDataset/train_list.txt',
    label_list='MyDataset/labels.txt',
    transforms=train_transforms,
    shuffle=True)
eval_dataset = pdx.datasets.VOCDetection(
    data_dir='MyDataset',
    file_list='MyDataset/val_list.txt',
    label_list='MyDataset/labels.txt',
    transforms=eval_transforms)

Initialize the model and train it
# can use VisualDL view the training target, the reference https://paddlex.readthedocs.io/zh_CN/develop/train/visualdl.html
num_classes = len(train_dataset.labels)

# API description: https://paddlex.readthedocs.io/zh_CN/develop/apis/models/detection.html#paddlex-det-yolov3
model = pdx.det.PPYOLO(num_classes=num_classes)

# API description: https://paddlex.readthedocs.io/zh_CN/develop/apis/models/detection.html#train
# of each parameter is introduced and adjust details: https://paddlex.readthedocs.io/zh_CN/develop/appendix/parameters.html
model.train(
    num_epochs=270,
    train_dataset=train_dataset,
    train_batch_size=8,
    eval_dataset=eval_dataset,
    learning_rate=0.000125,
    lr_decay_epochs=[210.240],
    save_dir='output/ppyolo',
    use_vdl=True)
Copy the code

Python deployment

PaddleX has integrated a high performance Python-based prediction interface, and here is a demonstration of the prediction effects of single images and video streams.

Derive prediction model

! paddlex --export_inference --model_dir=./output/ppyolo/best_model --save_dir=./inference_modelCopy the code

[09-14 00:02:43 MainThread @logger.py:224] Argv: /opt/conda/envs/python35-paddle120-env/bin/paddlex --export_inference --model_dir=./output/ppyolo/best_model --save_dir=./ inference_Model W0914 00:02:44.759682 6403 Device_context. cc:252] Please NOTE: Device :0, CUDA Capability: 70, Driver API Version: 9.2, Runtime API Version: 9.0 W0914 00:02:44.765246 6403 Device_context. cc:260] Device: 0, cuDNN Version: 7.6. / opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/paddle/fluid/IO py: 1998: UserWarning: This list is not set, Because of Paramerter not found in program. There are: create_parameter_0.w_0 create_parameter_1.w_0 create_parameter_2.w_0 create_parameter_3.w_0 create_parameter_4.w_0 create_parameter_5.w_0 create_parameter_6.w_0 create_parameter_7.w_0 create_parameter_8.w_0 create_parameter_9.w_0 create_parameter_10.w_0 create_parameter_11.w_0 create_parameter_12.w_0 create_parameter_13.w_0 create_parameter_14.w_0 create_parameter_15.w_0 create_parameter_16.w_0 create_parameter_17.w_0 create_parameter_18.w_0 create_parameter_19.w_0 create_parameter_20.w_0 create_parameter_21.w_0 create_parameter_22.w_0 create_parameter_23.w_0 create_parameter_24.w_0 create_parameter_25.w_0 create_parameter_26.w_0 create_parameter_27.w_0 create_parameter_28.w_0 create_parameter_29.w_0 create_parameter_30.w_0 create_parameter_31.w_0 create_parameter_32.w_0 create_parameter_33.w_0 create_parameter_34.w_0 create_parameter_35.w_0 create_parameter_36.w_0 create_parameter_37.w_0 create_parameter_38.w_0 create_parameter_39.w_0 create_parameter_40.w_0 create_parameter_41.w_0 create_parameter_42.w_0 create_parameter_43.w_0 create_parameter_44.w_0 create_parameter_45.w_0 create_parameter_46.w_0 create_parameter_47.w_0 format(" ".join(unused_para_list))) 2020-09-14 00:02:49 [INFO] Model[PPYOLO] loaded. 2020-09-14 00:02:51 [INFO] Model for inference deploy saved in ./inference_model.Copy the code

Single picture prediction

Select an image from the test set to see the prediction effect

import paddlex as pdx
predictor = pdx.deploy.Predictor('./inference_model')
result = predictor.predict(image='MyDataset/images/hard_hat_workers1457.png')
Copy the code

2020-09-14 00:03:07 [WARNING]	HRNet/DeepLabv3p/PPYOLO are not supported for the use of mkldnn
Copy the code

%matplotlib inline
import matplotlib.pyplot as plt # PLT is used to display images
import numpy as np
import cv2

# Read the original image
origin_pic = cv2.imread('MyDataset/images/hard_hat_workers1457.png')
origin_pic = cv2.cvtColor(origin_pic, cv2.COLOR_BGR2RGB)
plt.imshow(origin_pic)
plt.axis('off') # do not display the axis
plt.show()
Copy the code

/ opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/matplotlib/cbook/set py: 2349: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, And in 3.8 it will stop working if isinstance(obj, collections.iterator): / opt/conda envs/python35 - paddle120 - env/lib/python3.7 / site - packages/matplotlib/cbook/set py: 2366: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, And in 3.8 it will stop working return list(data) if isinstance(data, collections.mappingView) else dataCopy the code

result
Copy the code

[{' category_id: 1, 'bbox: [125.0842514038086, 0.0384979248046875, 136.88593292236328, 115.65148162841797],' score ': 0.9950428009033203, 'category': 'helmet'}, {'category_id': 1, 'bbox': [390.4376525878906, 27.37880516052246, 25.49139404296875, 45.28906440734863], score: 0.12913084030151367, 'category': 'Helmet '}, {'category_id': 1, 'bbox': [134.69830322265625, 0.0, 129.36032104492188, 115.4359130859375], 'score': 0.08461114019155502, 'category': 'helmet'}, {'category_id': 2, 'bbox': [154.33670043945312, 0.0, 236.26812744140625, 342.99615478515625], 'Score ': 0.07235386967658997, 'category': 'person'}, {'category_id': 1, 'bbox': [193.6173095703125, 188.1810760498047, 48.258544921875, 46.7227783203125], 'Score ': 0.06879084557294846, 'category': 'helmet'}, {'category_id': 2, 'bbox': [355.259033203125, 16.935226440429688, 55.0848388671875, 294.5183868408203], 'Score ': 0.05975065752863884, 'category': 'person'}, {'category_id': 1, 'bbox': [126.0989990234375, 3.00634765625, 132.97171020507812, 126.83741760253906], 'score': 0.05429061874747276, 'category': 'helmet'}, {'category_id': 1, 'bbox': [80.74116516113281, 112.00466918945312, 72.82310485839844, 93.2613525390625], 0.040811680257320404, 'category': 'helmet'}, {'category_id': 1, 'bbox': [408.301513671875, 244.8852081298828, 7.698486328125, 23.135787963867188], 'score': 0.03967300429940224, 'category': 'helmet'}, {'category_id': 1, 'bbox': [131.4346923828125, 4.035350799560547, 136.84912109375, 124.77803421020508], 'Score ': 0.03753501549363136, 'category': 'helmet'}, {'category_id': 1, 'bbox': [397.41314697265625, 23.68865966796875, 17.677734375, 49.96672058105469], 'score': 0.030422167852520943, 'category': 'helmet'}, {'category_id': 1, 'bbox': [354.68927001953125, 4.3401031494140625, 56.24041748046875, 279.62767028808594], 'score': 0.027957145124673843, 'category': 'helmet'}, {'category_id': 1, 'bbox': [384.8450927734375, 23.632007598876953, 30.70550537109375, 72.5086784362793], 'score': 0.025589050725102425, 'category': 'helmet'}, {'category_id': 1, 'bbox': [192.7164306640625, 97.08804321289062, 39.828460693359375, 38.87474060058594], 0.020570214837789536, 'category': 'helmet'}, {'category_id': 1, 'bbox': [329.08734130859375, 408.01483154296875, 26.87847900390625, 7.94439697265625], 'score': 0.01700388640165329, 'category': 'helmet'}, {'category_id': 2, 'bbox': [138.332763671875, 0.0, 229.27078247070312, 288.4775695800781], score: 0.015100134536623955, 'category': 'person'}, {'category_id': 2, 'bbox': [2.211437225341797, 61.75689697265625, 54.9390754699707, 332.4832763671875], 'score': 0.014021923765540123, 'category': 'person'}, {'category_id': 1, 'bbox': [2.211437225341797, 61.75689697265625, 54.9390754699707, 332.4832763671875], 'score': 0.013291412964463234, 'category': 'Helmet '}, {'category_id': 2, 'bbox': [103.25830078125, 0.0, 257.49166660156, 346.92462158203125], 'score': 0.013123809359967709, 'category': 'person'}, {'category_id': 1, 'bbox': [383.8271179199219, 18.029033660888672, 32.172882080078125, 67.19771194458008] 0.012730448506772518, 'category': 'helmet'}, {'category_id': 1, 'bbox': [383.1015930175781, 24.023815155029297, 32.49530029296875, 92.09848403930664], 'score': 0.01179821789264679, 'category': 'helmet'}, {'category_id': 2, 'bbox': [77.20639038085938, 0.0, 265.5789794921875, 348.062744140625], 'score': 0.01147290039807558, 'category': 'person'}, {'category_id': 2, 'bbox': [131.15541076660156, 0.0, 2000, 001], 'score': 0.010981513187289238, 'category': 'person'}, {'category_id': 2, 'bbox': [334.14312744140625, 8.3800048828125, 69.71087646484375, 310.58306884765625], 'score': 0.010134699754416943, 'category:' person '}]Copy the code

Video stream prediction

It is not possible to demonstrate real-time effects in AI Studio, so the prediction images are saved and then combined into a video. The paddlex.det.visualize() method also needs to be overridden

?? paddlex.det.visualizeCopy the code

def draw_bbox_mask(image, results, threshold=0.5) :
    import matplotlib
    matplotlib.use('Agg')
    import matplotlib as mpl
    import matplotlib.figure as mplfigure
    import matplotlib.colors as mplc
    from matplotlib.backends.backend_agg import FigureCanvasAgg

    # refer to https://github.com/facebookresearch/detectron2/blob/master/detectron2/utils/visualizer.py
    def _change_color_brightness(color, brightness_factor) :
        assert brightness_factor >= -1.0 and brightness_factor <= 1.0
        color = mplc.to_rgb(color)
        polygon_color = colorsys.rgb_to_hls(*mplc.to_rgb(color))
        modified_lightness = polygon_color[1] + (brightness_factor *
                                                 polygon_color[1])
        modified_lightness = 0.0 if modified_lightness < 0.0 else modified_lightness
        modified_lightness = 1.0 if modified_lightness > 1.0 else modified_lightness
        modified_color = colorsys.hls_to_rgb(
            polygon_color[0], modified_lightness, polygon_color[2])
        return modified_color

    _SMALL_OBJECT_AREA_THRESH = 1000
    # setup figure
    width, height = image.shape[1], image.shape[0]
    scale = 1
    fig = mplfigure.Figure(frameon=False)
    dpi = fig.get_dpi()
    fig.set_size_inches(
        (width * scale + 1e-2) / dpi,
        (height * scale + 1e-2) / dpi, )
    canvas = FigureCanvasAgg(fig)
    ax = fig.add_axes([0.0.0.0.1.0.1.0])
    ax.axis("off")
    ax.set_xlim(0.0, width)
    ax.set_ylim(height)
    default_font_size = max(np.sqrt(height * width) // 90.10 // scale)
    linewidth = max(default_font_size / 4.1)

    labels = list(a)for dt in np.array(results):
        if dt['category'] not in labels:
            labels.append(dt['category'])
    color_map = get_color_map_list(256)

    keep_results = []
    areas = []
    for dt in np.array(results):
        cname, bbox, score = dt['category'], dt['bbox'], dt['score']
        if score < threshold:
            continue
        keep_results.append(dt)
        areas.append(bbox[2] * bbox[3])
    areas = np.asarray(areas)
    sorted_idxs = np.argsort(-areas).tolist()
    keep_results = [keep_results[k]
                    for k in sorted_idxs] if len(keep_results) > 0 else []

    for dt in np.array(keep_results):
        cname, bbox, score = dt['category'], dt['bbox'], dt['score']
        xmin, ymin, w, h = bbox
        xmax = xmin + w
        ymax = ymin + h

        color = tuple(color_map[labels.index(cname) + 2])
        color = [c / 255. for c in color]
        # draw bbox
        ax.add_patch(
            mpl.patches.Rectangle(
                (xmin, ymin),
                w,
                h,
                fill=False,
                edgecolor=color,
                linewidth=linewidth * scale,
                alpha=0.8,
                linestyle="-")),# draw mask
        if 'mask' in dt:
            mask = dt['mask']
            mask = np.ascontiguousarray(mask)
            res = cv2.findContours(
                mask.astype("uint8"), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
            hierarchy = res[-1]
            alpha = 0.5
            if hierarchy is not None:
                has_holes = (hierarchy.reshape(-1.4) :3] > =0).sum(a) >0
                res = res[-2]
                res = [x.flatten() for x in res]
                res = [x for x in res if len(x) >= 6]
                for segment in res:
                    segment = segment.reshape(-1.2)
                    edge_color = mplc.to_rgb(color) + (1, )
                    polygon = mpl.patches.Polygon(
                        segment,
                        fill=True,
                        facecolor=mplc.to_rgb(color) + (alpha, ),
                        edgecolor=edge_color,
                        linewidth=max(default_font_size // 15 * scale, 1), )
                    ax.add_patch(polygon)

        # draw label
        text_pos = (xmin, ymin)
        horiz_align = "left"
        instance_area = w * h
        if (instance_area < _SMALL_OBJECT_AREA_THRESH * scale or
                h < 40 * scale):
            if ymin >= height - 5:
                text_pos = (xmin, ymin)
            else:
                text_pos = (xmin, ymax)
        height_ratio = h / np.sqrt(height * width)
        font_size = (np.clip((height_ratio - 0.02) / 0.08 + 1.1.2.2) * 0.5 * default_font_size)
        text = "{} {:.2f}".format(cname, score)
        color = np.maximum(list(mplc.to_rgb(color)), 0.2)
        color[np.argmax(color)] = max(0.8, np.max(color))
        color = _change_color_brightness(color, brightness_factor=0.7)
        ax.text(
            text_pos[0],
            text_pos[1],
            text,
            size=font_size * scale,
            family="sans-serif",
            bbox={
                "facecolor": "black"."alpha": 0.8."pad": 0.7."edgecolor": "none"
            },
            verticalalignment="top",
            horizontalalignment=horiz_align,
            color=color,
            zorder=10,
            rotation=0, )

    s, (width, height) = canvas.print_to_buffer()
    buffer = np.frombuffer(s, dtype="uint8")

    img_rgba = buffer.reshape(height, width, 4)
    rgb, alpha = np.split(img_rgba, [3], axis=2)

    try:
        import numexpr as ne
        visualized_image = ne.evaluate(
            "Image * (1-alpha / 255.0) + RGB * (alpha / 255.0)")
    except ImportError:
        alpha = alpha.astype("float32") / 255.0
        visualized_image = image * (1 - alpha) + rgb * alpha

    visualized_image = visualized_image.astype("uint8")

    return visualized_image
Copy the code

def get_color_map_list(num_classes) :
    """ Returns the color map for visualizing the segmentation mask, which can support arbitrary number of classes. Args: num_classes: Number of classes Returns: The color map """
    color_map = num_classes * [0.0.0]
    for i in range(0, num_classes):
        j = 0
        lab = i
        while lab:
            color_map[i * 3] |= (((lab >> 0) & 1) < < (7 - j))
            color_map[i * 3 + 1] |= (((lab >> 1) & 1) < < (7 - j))
            color_map[i * 3 + 2] |= (((lab >> 2) & 1) < < (7 - j))
            j += 1
            lab >>= 3
    color_map = [color_map[i:i + 3] for i in range(0.len(color_map), 3)]
    return color_map
Copy the code

def visualize(image, result, threshold=0.5, img_num=0,save_dir='/') :
    """ Visualize bbox and mask results """

    if isinstance(image, np.ndarray):
        image_name = str(img_num) + '.jpg'
    else:
        image_name = os.path.split(image)[-1]
        image = cv2.imread(image)

    image = draw_bbox_mask(image, result, threshold=threshold)
    if save_dir is not None:
        if not os.path.exists(save_dir):
            os.makedirs(save_dir)
        out_path = os.path.join(save_dir, '{}'.format(image_name))
        cv2.imwrite(out_path, image)
        print('The visualized result is saved as {}'.format(out_path))
    else:
        return image
Copy the code

import time
import os
import sys
import colorama
from colorama import init
import paddlex

init(autoreset=True)
levels = {0: 'ERROR'.1: 'WARNING'.2: 'INFO'.3: 'DEBUG'}


def log(level=2, message="", use_color=False) :
    current_time = time.time()
    time_array = time.localtime(current_time)
    current_time = time.strftime("%Y-%m-%d %H:%M:%S", time_array)
    if paddlex.log_level >= level:
        if use_color:
            print("\033[1;31;40m{} [{}]\t{}\033[0m".format(
                current_time, levels[level], message).encode("utf-8").decode(
                    "latin1"))
        else:
            print("{} [{}]\t{}".format(current_time, levels[level], message)
                  .encode("utf-8").decode("latin1"))
        sys.stdout.flush()


def debug(message="", use_color=False) :
    log(level=3, message=message, use_color=use_color)


def info(message="", use_color=False) :
    log(level=2, message=message, use_color=use_color)


def warning(message="", use_color=True) :
    log(level=1, message=message, use_color=use_color)


def error(message="", use_color=True, exit=True) :
    log(level=0, message=message, use_color=use_color)
    if exit:
        sys.exit(-1)

Copy the code

import cv2
import paddlex as pdx
import numpy as np
import colorsys
import os

predictor = pdx.deploy.Predictor('./inference_model')
cap = cv2.VideoCapture('./hatdet.mp4')
i = 1
while cap.isOpened():
    ret, frame = cap.read()
    if ret:
        result = predictor.predict(frame)
        print(i)
        vis_img = visualize(frame, result, threshold=0.4, img_num=i, save_dir='hatdet')
        i += 1
        The local environment can check the safety helmet detection effect in real time
        # cv2.imshow('hatdet', vis_img)
        if cv2.waitKey(1) & 0xFF= =ord('q') :break
    else:
        break
cap.release()
Copy the code

# Combine images into videos! ffmpeg -f image2 -i ./hatdet/%d.jpg -vcodec libx264 -r30 ppyolo.mp4
Copy the code

Ffmpeg Version 2.8.15-0Ubuntu0.16.04.1 Copyright (C) 2000-2018 The FFMPEG developers built with GCC 5.4.0 (Ubuntu 5.4.0-6 ubuntu1 ~ 16.04.10) 20160609 configuration: -- prefix = / usr - extra - version = 0 ubuntu0. 16.04.1 - build - suffix = - ffmpeg toolchain = hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared  --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv libavutil 54.31.100/54.31.100 libavCodec 56.60.100/56.60.100 libavformat 56.40.101/56.40.101 libavDevice 56.4.100 / 56. 4.100 libavFilter 5.40.101/5.40.101 libavresample 2.1.0/2. 0 libswScale 3.1.101/3 1.2.101/1.2.101 libpostproc 53.3.100/53.100 [mjpeg @0x12ba720] Changps to 8 Input #0, image2, From './hatdet/%d.jpg': Duration: 00:00:14.96, start: 0.000000, bitrate: N/A Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn, 25 tbc No pixel format specified, yuvj420p for H.264 encoding chosen. Use -pix_fmt yuv420p for compatibility with outdated media players. [libx264 @ 0x12bc620] using SAR=1/1 [libx264 @ 0x12bc620] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2 [libx264@0x12bc620] profile High Level 4.0 [libx264@0x12bc620] 264-core 148 r2643 5C65704 - H.264/ mPEG-4 AVC COdec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: Cabac =1 ref=3 deblock=1:0:0 ANALYSE =0x3:0x113 me=hex subme=7 PSY =1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 Deadzone =21,11 fast_pskip=1 chroma_qp_offset= -1 threads=34 lookahead_threads=5 sliced_threads=0  nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 CRF =23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 IP_ratio =1.40 aq=1:1.00 Output #0, mp4, to 'ppyolo. Mp4 ': Metadata: Encoder: Lavf56.40.101 Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 30 fps, 15360 tbn, 30 TBC Metadata: encoder: Lavc56.60.100 libx264 Stream Mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264)) Press [q] to stop, [?] for help frame= 449 FPS = 23 Q = -1.0lsize = 6329kB Time =00:00:14.90 bitRate =3479.9kbits/s DUP =75 DROP =0 Video :6324kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.085457% [libx264@0x12bc620] frame I:4 Avg QP:21.75 size: 67748 [libx264@0x12bc620] frame P:237 Avg QP:22.72 size: 22371 [libx264@0x12bc620] frame B: 2020AVg QP: 2020SIZE: 437 [libx264@0x12bc620] frame B: 2020avg QP: 2020size: 437 [libx264@0x12bc620] frame B-frames: [libx264@0x12bc620] MB I I16.. 4: 18.6% 71.8% 9.6% [libx264@0x12bc620] MB P I16.. 4: 4.2% 10.2% 0.9% P16.. [libx264@0x12bc620] MB B I16.. 4: 0.4% 0.6% 0.0% B16.. 8:0.7% 24.1% 1.7% 0.1% direct: skip: 72.4% L0:37.1% L1:60.2% BI: [libx264@0x12bc620] 8x8 Transform intra:66.6% Inter :77.7% [libx264@0x12bc620] coded Y,uvDC,uvAC intra: [libx264@0x12bc620] i16 v,h,dc,p: 36% 33% 10% 20% [libx264 @ 0x12bc620] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 24% 23% 3% 3% 4% 4% 4% 4% [libx264 @ 0x12bc620] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 25% 11% 4% 5% 6% 5% 4% 4% [libx264 @ 0x12bc620] i8c dc,h,v,p: [libx264@0x12bc620] Weighted p-frames: Y: 1.0% UV: 1.0% [libx264@0x12bc620] ref P L0: libx264@0x12bc620] Weighted p-frames: Y: 1.0% UV: 1.0% [libx264@0x12bc620] ref B L0: 1.0% [libx264@0x12bc620] ref B L0: 1.0% [libx264@0x12bc620] ref B L0: 1.0% [libx264@0x12bc620] ref B L0: 1.0% [libx264@0x12bc620] ref B L0: 1.0% [libx264@0x12bc620] KB /s:3461.08Copy the code

We can see that PP-YOLO actually recognizes ordinary hats as hard hats, and there is room for improvement compared to Yolov3_Darknet53.

import IPython
IPython.display.Video('ppyolo.mp4')
Copy the code

video

Model encryption Deployment (Linux platform)

Introduction to the

Source: PaddleX official documentation

(1) Selection of encryption algorithm and supported library

The OpenSSL library is generally used to support data encryption and decryption. OpenSSL provides a large number of encryption and decryption algorithms, including symmetric encryption algorithm (AES) and asymmetric encryption algorithm (RSA).

The two algorithms use different scenarios. The asymmetric encryption algorithm is generally applied to digital signature and key negotiation scenarios, while the symmetric encryption algorithm is generally applied to pure data encryption scenarios and has better performance. Symmetric encryption algorithm is used in the process of model encryption.

The author’s note: In simple terms, asymmetric encryption algorithm has better confidentiality and is much more difficult to crack by force than symmetric encryption algorithm, but this is at the cost of time. Asymmetric encryption algorithm is not suitable for large-scale data encryption.

The following description of the model encryption scenario is based on the development of a C/C++ library and the use of AES symmetric encryption algorithm. In order to quickly judge whether the decryption is successful before and after encryption and decryption, the use of AES-GCM encryption and decryption mode uses 256 bits of key data for the security of the key.

(2) General steps to realize model protection:

The following is a Description of the internal implementation of the PROVIDED C/C++ encryption and decryption library in Chinese. You can follow the steps below to implement a set of encryption and decryption libraries to suit your own scenario and load them into the Paddle Inference library through in-memory data

1) Considering the need to load data from memory after decrypting the encrypted model file, generate model file and parameter file using Conbine’s mode.

2) The project integrates OpenSSL, using the form of static library.

3) To implement AES algorithm interface, with the help of EVP interface provided by OpenSSL, specify algorithm type in EVP interface, algorithm uses AES of symmetric encryption and decryption algorithm, encryption and decryption mode uses AES-GCM, the key length is 256 bits, the implementation of AES-GCM can refer to the official example to encapsulate the interface: AES – GCM.

4) Use OpenSSL library to implement SHA256 digest algorithm, this part is useful below (optional). For details about the hash calculation of SHA256, see the Example provided by OpenSSL: OpenSSL Information Summary example.

5) In the model encryption link, the data content of model file and Params file is directly encrypted and saved to a new file. In order to distinguish and iterate the new file, in addition to the encrypted data, header information is added, such as the fixed magic number is used as the beginning of the file to determine the file type; In order to facilitate subsequent iterations, write the version number to distinguish; In order to determine whether the same key is used during decryption, the encryption key is stored after SHA256 calculation. These three parts make up the header information of the current encrypted file. The encrypted file contains header information and ciphertext information.

6) In the decryption process of the model, read the relevant encrypted data into the memory according to the encrypted file, and decrypt the memory data using AES algorithm. Note that the encryption algorithm and encryption mode, as well as the data and length of the key, shall be consistent with the encryption algorithm during decryption, otherwise the decrypted data will be wrong.

7) C/C++ library integrating model prediction, which usually involves paddle::AnalysisConfig and Paddle ::Predictor. In order to load the decrypted model plaintext data directly from the in-memory data (to avoid creating temporary files after model decryption), Here you need to replace the model loading function of AnalysisConfig from SetModel with SetModelBuffer to load model data from memory.

It is important to note that in this scenario, the key is integrated in the code of the upper predictive service. So the security of the model is equal to the strength of the code against reverse debugging. In order to protect key and model security, developers also need to harden their applications. Common application hardening means are: code obfuscation, binary file shell, etc., or change the encryption mechanism to AES white box encryption technology to protect the key. There are a large number of commercial and open source products available in this area of technology, which will not be covered here.

Download and unzip the encryption tool

Linux Version PaddleX model encryption tool. This version will be automatically downloaded when the script is compiled locally. However, the AI Studio compilation fails.

! wget https://bj.bcebos.com/paddlex/tools/1.2. 0/paddlex-encryption.zip
Copy the code

- the 2020-09-14 00:17:31 - Resolving bj.bcebos.com https://bj.bcebos.com/paddlex/tools/1.2.0/paddlex-encryption.zip (bj.bcebos.com)... 182.61.200.229 182.61.200.195, 2409:8 c00:6 c21:10 AD: 0: ff: b00e: 67 d Connecting to bj.bcebos.com (bj.bcebos.com) | 182.61.200.229 | : 443... connected. HTTP request sent, awaiting response... 200 OK Length: 3972031 (3.8m) [application/octet-stream] Saving to: 'paddlex - encryption.zip' paddlex - encryption. 100% [= = = = = = = = = = = = = = = = = = = >] 3.79 M 8.38 MB/s in 0.5 s 00:17:32 2020-09-14 (8.38) MB/s) - 'paddlex-encryption.zip' saved [3972031/3972031]Copy the code

! unzip paddlex-encryption.zip
Copy the code

Archive:  paddlex-encryption.zip
   creating: paddlex-encryption/
   creating: paddlex-encryption/include/
  inflating: paddlex-encryption/include/paddle_model_encrypt.h  
  inflating: paddlex-encryption/include/paddle_model_decrypt.h  
  inflating: paddlex-encryption/include/model_code.h  
  inflating: paddlex-encryption/.DS_Store  
  inflating: paddlex-encryption/README  
   creating: paddlex-encryption/tool/
  inflating: paddlex-encryption/tool/paddle_encrypt_tool  
   creating: paddlex-encryption/lib/
  inflating: paddlex-encryption/lib/libpmodel-encrypt.so  
  inflating: paddlex-encryption/lib/libpmodel-decrypt.so  
Copy the code

Export the PaddleX encryption model

After encryption is complete, the encrypted model is saved to the specified -save_dir, containing __model__.encrypted, __params__.encrypted, and model.yml files, and key information is generated. The key for onHEuCBj4kYLuRmDdPtbODXpvdYaVRLRB/eWhkopx8U =

! ./paddlex-encryption/tool/paddle_encrypt_tool -model_dir /home/aistudio/inference_model -save_dir /home/aistudio/paddlex_encrypted_model
Copy the code

Output: Encryption key: 
	onHEuCBj4kYLuRmDdPtbODXpvdYaVRLRB/eWhkopx8U=
Success, Encrypt __model__, __params__ to /home/aistudio/paddlex_encrypted_model(dir) success!
Copy the code

The local Linux platform deploys the encryption model

Note: The following needs to be done locally, one of the key reasons is that the GCC version on AI Studio is too high and there is no sudo permission to lower the GCC version.

Prerequisites required by the PaddleX document

• G + + 4.8.2 ~ 4.9.4
• CUDA 9.0 / CUDA 10.0, CUDNN 7+ (only required when using the GPU version of the prediction library)
• CMake 3.0 +

The deployment environment where the local tests are actually done

• G + + 4.8.5
• CUDA 10.2, CUDNN 7.6.5
• CMake 3.10.2
• Network environment

Install GCC

Reference: GCC downgrade and upgrade

The GCC version is too high in the local environment. An error occurred during compilation, so the GCC is degraded

1. Install gcc4.8

Sudo apt-get install -y GCC -4.8 sudo apt-get install -y g++-4.8Copy the code

2. Re-establish the soft connection

cd /usr/bin    Go to the /usr/bin folder
sudo rm -r gcc  Remove the previous soft connectionSudo ln -sf gcc-4.8gccEstablish gCC4.7 soft connection
sudo rm -r g++  # with GCCSudo ln -sf g++ -4.8g ++Copy the code

3. Verify the GCC version

gcc --version Check the GCC version
Copy the code

GCC (Ubuntu 4.8.5-4Ubuntu8) 4.8.5 Copyright (C) 2015 Free Software Foundation, Inc. This is Free Software; see thesource for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Copy the code

Install the Linux prediction library

1. Pull PaddleX model library

git clone https://gitee.com/paddlepaddle/PaddleX.git
Copy the code

Note: c + + prediction code in/root/projects/PaddleX/deploy/CPP directory, the directory is not dependent on any other directories under PaddleX.

2. Download PaddlePaddle C++ prediction library paddle_inference

Follow the PaddleX documentation and the PaddleX website instructions to choose the appropriate version to install

Note: nV-Jetson-CUDa10-CUDNnn7.5-TRt5 are all precompiled based on GCC 4.8.5. If you use older versions of GCC, there may be ABI compatibility issues. You are advised to downgrade or compile the prediction library yourself.

In this article, I used the Ubuntu14.04_CUDA10.0_CUDNn7_AVx_MKL prediction library (version 1.8.4)

After downloading the installation package, decompress it to the PaddleX/deploy/ CPP /fluid_inference directory. The contents are as follows:

Fluid_inference ├ ─ ─ paddle # paddle core libraries and header files | ├ ─ ─ third_party # third-party dependent libraries and header files | └ ─ ─ version. TXT # version and compile informationCopy the code

3. The compilation

After changing the working directory to the PaddleX/deploy/ CPP directory, simply execute the build script to start compiling.

sh scripts/build.sh
Copy the code

But! You need to configure ‘scripts/build.sh’ first, otherwise you will get an error when predicting! \ color # {FF3030} {but! You need to configure ‘scripts/build.sh’ first, otherwise you will get an error in the forecast! } but! You need to configure ‘scripts/build.sh’ first, otherwise you will get an error when predicting! The script comments are clear:

# Use GPU(i.e. CUDA or not)
WITH_GPU=OFF
Use MKL or openblas
WITH_MKL=ON
WITH_GPU=ON
WITH_TENSORRT=OFF
If you want to integrate TensorRT, change it to the TensorRT path you actually installed
TENSORRT_DIR=/root/projects/TensorRT/
# Paddle Prediction library path, please modify to the prediction library path that you actually installed
PADDLE_DIR=/root/projects/fluid_inference
Whether the prediction library of # Paddle is compiled using a static library
# When using TensorRT, the Paddle prediction library is usually a dynamic library
WITH_STATIC_LIB=OFF
# CUDA lib path
CUDA_LIB=/usr/local/cuda/lib64
# CUDNN lib path
CUDNN_LIB=/usr/local/cuda/lib64

Whether to load the encrypted model
WITH_ENCRYPTION=ON
The path of the encryption tool can not be modified if the pre-compiled version is used
sh $(pwd)/scripts/bootstrap.sh Download a precompiled version of the encryption tool
ENCRYPTION_DIR=$(pwd)/paddlex-encryption

# OPENCV path, if you use the built-in precompiled version can not be modified
sh $(pwd)/scripts/bootstrap.sh  Download a precompiled version of OpencV
OPENCV_DIR=$(pwd)/deps/opencv3gcc4. 8 /No changes are required below
rm -rf build
mkdir -p build
cd build
cmake .. \
    -DWITH_GPU=${WITH_GPU} \
    -DWITH_MKL=${WITH_MKL} \
    -DWITH_TENSORRT=${WITH_TENSORRT} \
    -DWITH_ENCRYPTION=${WITH_ENCRYPTION} \
    -DTENSORRT_DIR=${TENSORRT_DIR} \
    -DPADDLE_DIR=${PADDLE_DIR} \
    -DWITH_STATIC_LIB=${WITH_STATIC_LIB} \
    -DCUDA_LIB=${CUDA_LIB} \
    -DCUDNN_LIB=${CUDNN_LIB} \
    -DENCRYPTION_DIR=${ENCRYPTION_DIR} \
    -DOPENCV_DIR=${OPENCV_DIR}
make
Copy the code

Pp-yolo model using the MKLDNN prediction library will report errors in CPU prediction. Since the local environment is Conda, CUDA and CUDNN paths are manually configured and static library prediction is selected. The main modifications are as follows:

# Use GPU(i.e. CUDA or not)
WITH_GPU=ON
Use MKL or openblas
WITH_MKL=ON
# no integration with TensorRT(WITH_GPU=ON)
WITH_TENSORRT=OFF
If you want to integrate TensorRT, change it to the TensorRT path you actually installed
TENSORRT_DIR=$(pwd)/TensorRT/
# Paddle Prediction library path, please modify to the prediction library path that you actually installed
PADDLE_DIR=$(pwd)/fluid_inference
The prediction library for # Paddle is compiled using a static library
WITH_STATIC_LIB=ON
# CUDA lib path, using miniconda environment absolute path, for reference
CUDA_LIB=/home/aistudio/miniconda3/envs/paddle/lib
# CUDNN lib path, using miniconda environment absolute path, for reference
CUDNN_LIB=/home/aistudio/miniconda3/envs/paddle/lib
Copy the code

At this point, you can execute the script and wait for compilation to complete:

Note: OPENCV, paddlex-Encryption, and YAML are automatically downloaded for Linux compilation. If the compilation environment cannot access the Internet, you can manually download them:

• Opencv3.4.6 gcc4.8 ffmpeg. Tar. Gz2
• paddlex-encryption.zip
• yaml-cpp.zip

The opencv3gcc4.8.tar.bz2 file is downloaded and decompressed. In script/build.sh, specify OPENCE_DIR as the decompressed directory.

The paddlex-encryption.zip file is downloaded and decompressed, then specify ENCRYPTION_DIR as the decompressed path in script/build.sh.

Yaml – CPP.zip file download without after decompression, the cmake/yaml cmake will URL in the URL https://bj.bcebos.com/paddlex/deploy/deps/yaml-cpp.zip, the path to download files.

Modify the script after setting the main parameters, execute the build script:

sh ./scripts/build.sh
Copy the code

Prediction and Visualization

1. Download the paddlex_encrypted_model directory and decompress it

• After successful compilation, the executable program of target detection picture prediction demo isbuild/demo/detectorThe main command parameters are described as follows:

parameter	instructions
model_dir	Path of the exported prediction model
image	The image file path to predict
image_list	A. TXT file that stores the image path by line
use_gpu	Whether to use GPU prediction, 0 or 1 is supported (default is 0)
use_trt	Whether to use TensorRT prediction, 0 or 1 supported (default is 0)
use_mkl	Whether to use MKL to speed up CPU prediction, 0 or 1 is supported (default is 1)
mkl_thread_num	Number of MKL reasoning threads. The default value is number of CPU processors
gpu_id	GPU device ID. The default value is 0
save_dir	Path to save visual results, default is “output”,Classfier Does not have this parameter
key	Key information generated during encryption. The default value “” indicates that the unencrypted model is loaded
batch_size	Predicted batch size, default is 1
thread_num	Predicted number of threads, default to number of CPU processors

• After successful compilation, the executable program of target detection video prediction demo isbuild/demo/video_detectorThe main command parameters are described as follows:

parameter	instructions
model_dir	Path of the exported prediction model
use_camera	Whether to use camera prediction, 0 or 1 is supported (default is 0)
camera_id	Camera device ID. The default value is 0
video_path	Path to the video file
use_gpu	Whether to use GPU prediction, 0 or 1 is supported (default is 0)
use_trt	Whether to use TensorRT prediction, 0 or 1 supported (default is 0)
use_mkl	Whether to use MKL to speed up CPU prediction, 0 or 1 is supported (default is 1)
mkl_thread_num	Number of MKL reasoning threads. The default value is number of CPU processors
gpu_id	GPU device ID. The default value is 0
show_result	When making predictions for video files, whether to display prediction visualization results in real time on the screen (because delay processing is added, the display results cannot reflect the real frame rate), support value 0 or 1(default value 0)
save_result	Whether to save the predicted visual results of each frame as a video file, with a supported value of 0 or 1(default is 1)
save_dir	Path to save visual results. Default value is “output”
key	Key information generated during encryption. The default value “” indicates that the unencrypted model is loaded

Note: If the system has no GUI, do not set show_result to 1. When using camera prediction, pressESCKey to turn off the camera and launch the predictive program.

2. Test the encryption effect

• Key error information is not provided

• Description Incorrect Key information is displayed

3. Image prediction effect

Predict when specified encryption model is derived when the correct key onHEuCBj4kYLuRmDdPtbODXpvdYaVRLRB/eWhkopx8U =, predict the command of the single image and the results as follows (note that the current directory to switch to PaddleX/deploy/CPP) :

./build/demo/detector --model_dir=./paddlex_encrypted_model --image=./hard_hat_workers1025.png --key=onHEuCBj4kYLuRmDdPtbODXpvdYaVRLRB/eWhkopx8U= --use_gpu=1 --save_dir=output --batch_size=2 --thread_num=2
Copy the code

HubServing Lightweight service-oriented deployment

Note: To deploy in this way, you need to ensure that the PaddleHub version in your Python environment is higher than 1.8.0, so you need to upgrade PaddleHub in AI Studio.

• Lightweight service-oriented deployment
• PaddleHub-Serving

Preparing the deployment model

The format of the deployment model is __model__, __params__, and model.yml files in the inference_Model directory.

! pip install paddlehub -UCopy the code

Model transformation

To convert PaddleX’s Inference Model to PaddleHub’s pre-training Model, one-click conversion can be done by using the command HUB convert, which is described as follows:

parameter	use
–model_dir/-m	PaddleX Inference ModelDirectory where
–module_name/-n	Generates the name of the pretraining model
–module_version/-v	Generates a version of the pretrained model, default to1.0.0
–output_dir/-o	The location where the pre-training model is generated. The default is{module_name}_{timestamp}

! hub convert --model_dir inference_model \ --module_name hatdet \ --module_version1.0 
Copy the code

The converted module is stored in `hatdet_1600013222.3970025`.
Copy the code

Model installation

The pre-training model compressed package in. Tar. gz format obtained after model conversion needs to be installed on the local computer before deployment. Run the hub install command to install the package

! hub install hatdet_16000132223970025./hatdet.tar.gz
Copy the code

[09-14 00:07:41 MainThread @logger.py:224] Argv: / opt/conda envs/python35 - paddle120 - env/bin/hub install hatdet_1600013222. 3970025 / hatdet. Tar. Gz 00:07:41 2020-09-14 [WARNING] HRNet/DeepLabv3p/PPYOLO are not supported for the use of mkldnn Successfully installed hatdetCopy the code

Deployment model

Open terminal 1 and type Hub Serving start -m hatdet to complete the one-click deployment of the helmet detection model

Prediction results and post-processing effect display

It can be seen that compared with the prediction effect of Yolov3_Darknet53, PP-YOLO may use a large number of tricks on NMS, resulting in a large number of output results.

# coding: utf8
%matplotlib inline
import requests
import json
import cv2
import base64
import numpy as np
import colorsys
import warnings
warnings.filterwarnings("ignore")
plt.figure(figsize=(12.8))

def cv2_to_base64(image) :
    data = cv2.imencode('.png', image)[1]
    return base64.b64encode(data.tostring()).decode('utf8')


if __name__ == '__main__':
    Get the base64 encoding of the image
    img1 = cv2_to_base64(cv2.imread("MyDataset/images/hard_hat_workers1957.png"))
    img2 = cv2_to_base64(cv2.imread("MyDataset/images/hard_hat_workers1457.png"))
    data = {'images': [img1, img2]}
    # data = {'images': [img1]}
    # to specify the content-type
    headers = {"Content-type": "application/json"}
    Send an HTTP request
    url = "http://127.0.0.1:8866/predict/hatdet"
    r = requests.post(url=url, headers=headers, data=json.dumps(data))

    R.son ()["results"][0] r.son ()["results"][0]
    print(r.json()["results"])
    # Visualize () using the rewrite Visualize () method to complete a post-processing of the forecast result
    # Show the prediction effect of the first image
    image = visualize(cv2.imread('MyDataset/images/hard_hat_workers1957.png'),r.json()["results"] [0], save_dir=None)
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    plt.imshow(image)
    plt.axis('off') # do not display the axis
    plt.show()
Copy the code

[[{'bbox': [331.7291564941406, 229.62734985351562, 34.57830810546875, 34.630523681640625], 'category': 'helmet', 'category_id': 1, 'score': 0.9967609643936157}, {'bbox': [170.50559997558594, 186.35784912109375, 51.22918701171875, 60.973846435546875], 'category': 'helmet', 'category_id': 1, 'score': 0.9955191612243652}, {'bbox': [367.94952392578125, 237.93905639648438, 47.58758544921875, 46.95306396484375], 'category': 'helmet', 'category_id': 1, 'score': 0.9948139786720276}, {'bbox': [269.2464904785156, 221.2207794189453, 12.7646484375, 14.504791259765625], 'category': 'helmet', 'category_id': 1, 'score': 0.9134480357170105}, {'bbox': [247.70266723632812, 216.89828491210938, 14.026214599609375, 17.512542724609375], 'category': 'helmet', 'category_id': 1, 'score': 0.630147397518158}, {'bbox': [371.85125732421875, 400.6864929199219, 39.05401611328125, 15.313507080078125], 'category': 'helmet', 'category_id': 1, 'score': 0.32807663083076477}, {'bbox': [232.36773681640625, 250.36880493164062, 9.77496337890625, 11.038238525390625], 'category': 'helmet', 'category_id': 1, 'score': 0.22743859887123108}, {'bbox': [247.68785095214844, 215.86863708496094, 13.522872924804688, 16.386444091796875], 'category': 'helmet', 'category_id': 1, 'score': 0.12421621382236481}, {'bbox': [170.62782287597656, 188.18780517578125, 54.709503173828125, 57.154815673828125], 'category': 'helmet', 'category_id': 1, 'score': 0.11123937368392944}, {'bbox': [332.8782958984375, 230.11276245117188, 36.01025390625, 34.94927978515625], 'category': 'helmet', 'category_id': 1, 'score': 0.1104872077703476}, {'bbox': [368.8426818847656, 239.85720825195312, 45.56915283203125, 47.18389892578125], 'category': 'helmet', 'category_id': 1, 'score': 0.09753906726837158}, {'bbox': [333.7652282714844, 228.7819366455078, 29.8846435546875, 36.16294860839844], 'category': 'helmet', 'category_id': 1, 'score': 0.09440983831882477}, {'bbox': [367.9332275390625, 235.57919311523438, 46.40655517578125, 51.41015625], 'category': 'helmet', 'category_id': 1, 'score': 0.08461157232522964}, {'bbox': [172.2590789794922, 185.5328369140625, 51.270599365234375, 61.94378662109375], 'category': 'helmet', 'category_id': 1, 'score': 0.07757699489593506}, {'bbox': [169.13418579101562, 186.46913146972656, 54.66619873046875, 60.063446044921875], 'category': 'helmet', 'category_id': 1, 'score': 0.07465662062168121}, {'bbox': [270.36102294921875, 222.24411010742188, 10.78363037109375, 11.933349609375], 'category': 'helmet', 'category_id': 1, 'score': 0.06554000824689865}, {'bbox': [267.4338073730469, 220.87721252441406, 13.05267333984375, 14.96063232421875], 'category': 'helmet', 'category_id': 1, 'score': 0.06357312202453613}, {'bbox': [245.9558563232422, 215.8375244140625, 16.916488647460938, 16.320526123046875], 'category': 'helmet', 'category_id': 1, 'score': 0.051948659121990204}, {'bbox': [368.4364318847656, 390.9608154296875, 44.38427734375, 25.0391845703125], 'category': 'helmet', 'category_id': 1, 'score': 0.051140397787094116}, {'bbox': [367.8598327636719, 397.6152648925781, 45.7178955078125, 17.5845947265625], 'category': 'helmet', 'category_id': 1, 'score': 0.05086497962474823}, {'bbox': [368.85870361328125, 401.0027160644531, 36.8643798828125, 14.997283935546875], 'category': 'helmet', 'category_id': 1, 'score': 0.04898037761449814}, {'bbox': [170.01132202148438, 187.3490447998047, 51.967681884765625, 64.31814575195312], 'category': 'helmet', 'category_id': 1, 'score': 0.048814401030540466}, {'bbox': [372.4987487792969, 239.1075897216797, 43.501251220703125, 44.19142150878906], 'category': 'helmet', 'category_id': 1, 'score': 0.046813417226076126}, {'bbox': [267.45770263671875, 220.89503479003906, 16.17913818359375, 14.983673095703125], 'category': 'helmet', 'category_id': 1, 'score': 0.045761942863464355}, {'bbox': [368.425048828125, 237.93093872070312, 46.38629150390625, 51.8974609375], 'category': 'helmet', 'category_id': 1, 'score': 0.04393104463815689}, {'bbox': [384.0188903808594, 402.9225158691406, 31.981109619140625, 13.077484130859375], 'category': 'helmet', 'category_id': 1, 'score': 0.043700143694877625}, {'bbox': [366.7271728515625, 264.3626708984375, 13.36773681640625, 16.06512451171875], 'category': 'helmet', 'category_id': 1, 'score': 0.04359801486134529}, {'bbox': [102.40280151367188, 348.91375732421875, 81.35186767578125, 66.049072265625], 'category': 'person', 'category_id': 2, 'score': 0.03943166509270668}, {'bbox': [372.8295593261719, 401.8208312988281, 43.170440673828125, 14.179168701171875], 'category': 'helmet', 'category_id': 1, 'score': 0.0348227322101593}, {'bbox': [133.96005249023438, 191.46456909179688, 104.44491577148438, 159.8836669921875], 'category': 'person', 'category_id': 2, 'score': 0.03477822244167328}, {'bbox': [246.3191680908203, 217.1565704345703, 16.919876098632812, 17.332733154296875], 'category': 'helmet', 'category_id': 1, 'score': 0.03387150168418884}, {'bbox': [365.52801513671875, 407.37603759765625, 31.73785400390625, 8.5118408203125], 'category': 'helmet', 'category_id': 1, 'score': 0.031664490699768066}, {'bbox': [168.27688598632812, 188.17425537109375, 55.204193115234375, 62.30096435546875], 'category': 'helmet', 'category_id': 1, 'score': 0.0294905137270689}, {'bbox': [363.9883728027344, 407.4378967285156, 24.6961669921875, 8.562103271484375], 'category': 'helmet', 'category_id': 1, 'score': 0.027937332168221474}, {'bbox': [231.36537170410156, 249.39035034179688, 11.917694091796875, 13.37841796875], 'category': 'helmet', 'category_id': 1, 'score': 0.027002258226275444}, {'bbox': [232.2095489501953, 347.88238525390625, 84.66212463378906, 62.21612548828125], 'category': 'person', 'category_id': 2, 'score': 0.026107603684067726}, {'bbox': [324.2493591308594, 350.9742736816406, 77.60015869140625, 61.277099609375], 'category': 'person', 'category_id': 2, 'score': 0.02461632341146469}, {'bbox': [237.1014404296875, 219.13815307617188, 41.24969482421875, 75.4588623046875], 'category': 'person', 'category_id': 2, 'score': 0.024424299597740173}, {'bbox': [170.20843505859375, 188.14755249023438, 56.268096923828125, 63.1717529296875], 'category': 'helmet', 'category_id': 1, 'score': 0.023364698514342308}, {'bbox': [172.5839080810547, 189.1525421142578, 45.856842041015625, 51.449066162109375], 'category': 'helmet', 'category_id': 1, 'score': 0.022948715835809708}, {'bbox': [115.38738250732422, 382.8497619628906, 49.516456604003906, 33.150238037109375], 'category': 'helmet', 'category_id': 1, 'score': 0.022707432508468628}, {'bbox': [363.2900085449219, 402.9479675292969, 37.55023193359375, 13.052032470703125], 'category': 'helmet', 'category_id': 1, 'score': 0.02220369502902031}, {'bbox': [265.3883056640625, 220.30307006835938, 16.45806884765625, 15.872589111328125], 'category': 'helmet', 'category_id': 1, 'score': 0.021186893805861473}, {'bbox': [379.47918701171875, 406.71466064453125, 33.957763671875, 9.264892578125], 'category': 'helmet', 'category_id': 1, 'score': 0.020644597709178925}, {'bbox': [386.0550842285156, 406.64495849609375, 29.944915771484375, 9.35504150390625], 'category': 'helmet', 'category_id': 1, 'score': 0.01910533383488655}, {'bbox': [268.72210693359375, 222.58937072753906, 13.50665283203125, 15.773101806640625], 'category': 'helmet', 'category_id': 1, 'score': 0.018787415698170662}, {'bbox': [353.57183837890625, 388.6909484863281, 55.6737060546875, 26.35528564453125], 'category': 'helmet', 'category_id': 1, 'score': 0.018766868859529495}, {'bbox': [356.8499755859375, 232.96954345703125, 57.8121337890625, 50.662353515625], 'category': 'helmet', 'category_id': 1, 'score': 0.01859154738485813}, {'bbox': [363.1852722167969, 403.48388671875, 26.12835693359375, 12.51611328125], 'category': 'helmet', 'category_id': 1, 'score': 0.01783447340130806}, {'bbox': [290.3291931152344, 215.76742553710938, 14.1812744140625, 15.773529052734375], 'category': 'helmet', 'category_id': 1, 'score': 0.017611123621463776}, {'bbox': [172.4273223876953, 187.08935546875, 51.127197265625, 64.940185546875], 'category': 'helmet', 'category_id': 1, 'score': 0.017321070656180382}, {'bbox': [0.3208489418029785, 64.30500793457031, 15.686517238616943, 39.35881042480469], 'category': 'helmet', 'category_id': 1, 'score': 0.015773454681038857}, {'bbox': [316.22607421875, 387.7694091796875, 41.59527587890625, 26.81256103515625], 'category': 'helmet', 'category_id': 1, 'score': 0.015681810677051544}, {'bbox': [249.45901489257812, 217.05238342285156, 10.442474365234375, 12.26776123046875], 'category': 'helmet', 'category_id': 1, 'score': 0.01517727691680193}, {'bbox': [370.8968505859375, 406.8766784667969, 31.53265380859375, 8.7171630859375], 'category': 'helmet', 'category_id': 1, 'score': 0.014667780138552189}, {'bbox': [270.1673278808594, 348.6383056640625, 95.533447265625, 63.4222412109375], 'category': 'person', 'category_id': 2, 'score': 0.014631879515945911}, {'bbox': [116.70548248291016, 201.69520568847656, 13.192512512207031, 15.981475830078125], 'category': 'helmet', 'category_id': 1, 'score': 0.014438899233937263}, {'bbox': [375.06927490234375, 395.8373718261719, 32.03741455078125, 18.49078369140625], 'category': 'helmet', 'category_id': 1, 'score': 0.013648224994540215}, {'bbox': [19.760963439941406, 347.0207824707031, 115.94335174560547, 68.97921752929688], 'category': 'person', 'category_id': 2, 'score': 0.013385860249400139}, {'bbox': [374.796875, 390.62939453125, 41.203125, 25.37060546875], 'category': 'helmet', 'category_id': 1, 'score': 0.013299377635121346}, {'bbox': [19.716651916503906, 103.9334716796875, 70.9515151977539, 71.8316650390625], 'category': 'helmet', 'category_id': 1, 'score': 0.013239901512861252}, {'bbox': [373.0161437988281, 398.7494812011719, 42.983856201171875, 16.73736572265625], 'category': 'helmet', 'category_id': 1, 'score': 0.013094686903059483}, {'bbox': [5.5061187744140625, 0.0, 96.74745178222656, 57.1971435546875], 'category': 'helmet', 'category_id': 1, 'score': 0.012763410806655884}, {'bbox': [131.44741821289062, 351.1435546875, 108.8992919921875, 63.92431640625], 'category': 'person', 'category_id': 2, 'score': 0.01250837929546833}, {'bbox': [0.0, 93.92575073242188, 16.77104949951172, 42.69465637207031], 'category': 'helmet', 'category_id': 1, 'score': 0.01228792779147625}, {'bbox': [152.2611083984375, 352.98480224609375, 112.86758422851562, 61.83294677734375], 'category': 'person', 'category_id': 2, 'score': 0.01219321321696043}, {'bbox': [257.7774658203125, 389.6871337890625, 26.43695068359375, 14.680908203125], 'category': 'helmet', 'category_id': 1, 'score': 0.01167545560747385}, {'bbox': [0.0, 345.23956298828125, 71.78589630126953, 70.76043701171875], 'category': 'person', 'category_id': 2, 'score': 0.011647245846688747}, {'bbox': [365.8010559082031, 409.3409118652344, 18.1397705078125, 6.659088134765625], 'category': 'helmet', 'category_id': 1, 'score': 0.011373327113687992}, {'bbox': [264.97796630859375, 223.1400146484375, 24.737060546875, 49.536163330078125], 'category': 'person', 'category_id': 2, 'score': 0.011219870299100876}, {'bbox': [351.8800048828125, 398.3606872558594, 58.69012451171875, 17.25238037109375], 'category': 'helmet', 'category_id': 1, 'score': 0.011159008368849754}, {'bbox': [113.02017211914062, 345.0218505859375, 50.101287841796875, 67.587646484375], 'category': 'person', 'category_id': 2, 'score': 0.011076455004513264}, {'bbox': [61.051780700683594, 200.8598175048828, 14.393890380859375, 17.901214599609375], 'category': 'helmet', 'category_id': 1, 'score': 0.010802910663187504}, {'bbox': [252.25814819335938, 215.00096130371094, 13.63568115234375, 17.0357666015625], 'category': 'helmet', 'category_id': 1, 'score': 0.010301719419658184}, {'bbox': [250.7511749267578, 222.4567413330078, 37.76744079589844, 70.62098693847656], 'category': 'person', 'category_id': 2, 'score': 0.010208141058683395}, {'bbox': [178.3153076171875, 352.6536865234375, 107.88491821289062, 61.86126708984375], 'category': 'person', 'category_id': 2, 'score': 0.010132195428013802}], [{'bbox': [125.0842514038086, 0.0384979248046875, 136.88593292236328, 115.65148162841797], 'category': 'helmet', 'category_id': 1, 'score': 0.9950428009033203}, {'bbox': [390.4376525878906, 27.37880516052246, 25.49139404296875, 45.28906440734863], 'category': 'helmet', 'category_id': 1, 'score': 0.12913084030151367}, {'bbox': [134.69830322265625, 0.0, 129.36032104492188, 115.4359130859375], 'category': 'helmet', 'category_id': 1, 'score': 0.08461114019155502}, {'bbox': [154.33670043945312, 0.0, 236.26812744140625, 342.99615478515625], 'category': 'person', 'category_id': 2, 'score': 0.07235386967658997}, {'bbox': [193.6173095703125, 188.1810760498047, 48.258544921875, 46.7227783203125], 'category': 'helmet', 'category_id': 1, 'score': 0.06879084557294846}, {'bbox': [355.259033203125, 16.935226440429688, 55.0848388671875, 294.5183868408203], 'category': 'person', 'category_id': 2, 'score': 0.05975065752863884}, {'bbox': [126.0989990234375, 3.00634765625, 132.97171020507812, 126.83741760253906], 'category': 'helmet', 'category_id': 1, 'score': 0.05429061874747276}, {'bbox': [80.74116516113281, 112.00466918945312, 72.82310485839844, 93.2613525390625], 'category': 'helmet', 'category_id': 1, 'score': 0.040811680257320404}, {'bbox': [408.301513671875, 244.8852081298828, 7.698486328125, 23.135787963867188], 'category': 'helmet', 'category_id': 1, 'score': 0.03967300429940224}, {'bbox': [131.4346923828125, 4.035350799560547, 136.84912109375, 124.77803421020508], 'category': 'helmet', 'category_id': 1, 'score': 0.03753501549363136}, {'bbox': [397.41314697265625, 23.68865966796875, 17.677734375, 49.96672058105469], 'category': 'helmet', 'category_id': 1, 'score': 0.030422167852520943}, {'bbox': [354.68927001953125, 4.3401031494140625, 56.24041748046875, 279.62767028808594], 'category': 'helmet', 'category_id': 1, 'score': 0.027957145124673843}, {'bbox': [384.8450927734375, 23.632007598876953, 30.70550537109375, 72.5086784362793], 'category': 'helmet', 'category_id': 1, 'score': 0.025589050725102425}, {'bbox': [192.7164306640625, 97.08804321289062, 39.828460693359375, 38.87474060058594], 'category': 'helmet', 'category_id': 1, 'score': 0.020570214837789536}, {'bbox': [329.08734130859375, 408.01483154296875, 26.87847900390625, 7.94439697265625], 'category': 'helmet', 'category_id': 1, 'score': 0.01700388640165329}, {'bbox': [138.332763671875, 0.0, 229.27078247070312, 288.4775695800781], 'category': 'person', 'category_id': 2, 'score': 0.015100134536623955}, {'bbox': [2.211437225341797, 61.75689697265625, 54.9390754699707, 332.4832763671875], 'category': 'person', 'category_id': 2, 'score': 0.014021923765540123}, {'bbox': [2.211437225341797, 61.75689697265625, 54.9390754699707, 332.4832763671875], 'category': 'helmet', 'category_id': 1, 'score': 0.013291412964463234}, {'bbox': [103.25830078125, 0.0, 257.4916076660156, 346.92462158203125], 'category': 'person', 'category_id': 2, 'score': 0.013123809359967709}, {'bbox': [383.8271179199219, 18.029033660888672, 32.172882080078125, 67.19771194458008], 'category': 'helmet', 'category_id': 1, 'score': 0.012730448506772518}, {'bbox': [383.1015930175781, 24.023815155029297, 32.49530029296875, 92.09848403930664], 'category': 'helmet', 'category_id': 1, 'score': 0.01179821789264679}, {'bbox': [77.20639038085938, 0.0, 265.5789794921875, 348.062744140625], 'category': 'person', 'category_id': 2, 'score': 0.01147290039807558}, {'bbox': [131.15541076660156, 0.0, 243.6357879638672, 341.47149658203125], 'category': 'person', 'category_id': 2, 'score': 0.010981513187289238}, {'bbox': [334.14312744140625, 8.3800048828125, 69.71087646484375, 310.58306884765625], 'category': 'person', 'category_id': 2, 'score': 0.010134699754416943}]]
Copy the code