DJL target detection Demo

An overview of the

Deep Java Library is a Deep learning Java Library launched by AWS in 2019. It already supports MXNet, PyTorch and TensorFlow models for training and reasoning. DJL is not tied to a fixed deep learning framework, so the same set of code can be adapted to different deep learning frameworks.

Here, according to the tutorial given by the official website, how to build the Demo of target detection is introduced. The realized functions include reading local pictures, loading the pre-training Model provided by the official Model Zoo, Model reasoning, and output the result graph of target detection to the local. Resources include: official tutorial on SSD model reasoning, Maven dependency configuration, DJL Maven BOM configuration, DJL version dependency collocation.

Engineering structures,

Creating a New Maven Project

The JDK > = 1.8

The project structure

Depend on the introduction of

Introduce DJL itself as well as other packages that DJL relies on.

    <build>
        <plugins>
            <! --> Set maven to compile using jdk8
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <! --> Manage the version of dependent package by BOM
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>ai.djl</groupId>
                <artifactId>bom</artifactId>
                <version>0.9.0</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>commons-cli</groupId>
            <artifactId>commons-cli</artifactId>
            <version>1.4</version>
        </dependency>
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
            <version>2.8.5</version>
        </dependency>
        <! --> Log dependencies <-->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>2.12.1</version>
        </dependency>
        <! --> Dependency packages that must be introduced to use DJL
        <dependency>
            <groupId>ai.djl</groupId>
            <artifactId>api</artifactId>
        </dependency>
        <! Apache Mxnet Engine Implementation <-->
        <dependency>
            <groupId>ai.djl.mxnet</groupId>
            <artifactId>mxnet-engine</artifactId>
        </dependency>
        <! Apache MXNet Native Library uses different deep learning framework models to introduce different dependency packages.
        <dependency>
            <groupId>ai.djl.mxnet</groupId>
            <artifactId>mxnet-native-auto</artifactId>
            <scope>runtime</scope>
        </dependency>
        <! A ModelZoo containing models exported from Apache MXNet<-->
        <dependency>
            <groupId>ai.djl.mxnet</groupId>
            <artifactId>mxnet-model-zoo</artifactId>
        </dependency>
    </dependencies>
Copy the code

Log configuration file

There are no fixed requirements for this part. You can configure it according to actual needs. Here we use a simple configuration file log4j2.xml found on the Internet


      
<Configuration status="WARN">
    <Properties>
        <property name="log_level" value="info" />
        <Property name="log_dir" value="log" />
        <property name="log_pattern"
                  value="[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%p] - [%t] %logger - %m%n" />
        <property name="file_name" value="test" />
        <property name="every_file_size" value="100 MB" />
    </Properties>
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="${log_pattern}" />
        </Console>
        <RollingFile name="RollingFile"
                     filename="${log_dir}/${file_name}.log"
                     filepattern="${log_dir}/$${date:yyyy-MM}/${file_name}-%d{yyyy-MM-dd}-%i.log">
            <ThresholdFilter level="DEBUG" onMatch="ACCEPT"
                             onMismatch="DENY" />
            <PatternLayout pattern="${log_pattern}" />
            <Policies>
                <SizeBasedTriggeringPolicy
                        size="${every_file_size}" />
                <TimeBasedTriggeringPolicy modulate="true"
                                           interval="1" />
            </Policies>
            <DefaultRolloverStrategy max="20" />
        </RollingFile>

        <RollingFile name="RollingFileErr"
                     fileName="${log_dir}/${file_name}-warnerr.log"
                     filePattern="${log_dir}/$${date:yyyy-MM}/${file_name}-%d{yyyy-MM-dd}-warnerr-%i.log">
            <ThresholdFilter level="WARN" onMatch="ACCEPT"
                             onMismatch="DENY" />
            <PatternLayout pattern="${log_pattern}" />
            <Policies>
                <SizeBasedTriggeringPolicy
                        size="${every_file_size}" />
                <TimeBasedTriggeringPolicy modulate="true"
                                           interval="1" />
            </Policies>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Root level="${log_level}">
            <AppenderRef ref="Console" />
            <AppenderRef ref="RollingFile" />
            <appender-ref ref="RollingFileErr" />
        </Root>
    </Loggers>
</Configuration>
Copy the code

Business code

Quote: github.com/awslabs/djl…

package org.town;

import ai.djl.Application;
import ai.djl.ModelException;
import ai.djl.engine.Engine;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelZoo;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.training.util.ProgressBar;
import ai.djl.translate.TranslateException;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/** * An example of inference using an object detection model. */
public final class ObjectDetection {

    private static final Logger logger = LoggerFactory.getLogger(ObjectDetection.class);

    private ObjectDetection(a) {}

    public static void main(String[] args) throws IOException, ModelException, TranslateException {
        DetectedObjects detection = ObjectDetection.predict();
        logger.info("{}", detection);
    }

    public static DetectedObjects predict(a) throws IOException, ModelException, TranslateException {
        Path imageFile = Paths.get("src/test/resources/dog_bike_car.jpg");
        Image img = ImageFactory.getInstance().fromFile(imageFile);

        String backbone;
        if ("TensorFlow".equals(Engine.getInstance().getEngineName())) {
            backbone = "mobilenet_v2";
        } else {
            backbone = "resnet50";
        }

        Criteria<Image, DetectedObjects> criteria =
                Criteria.builder()
                        .optApplication(Application.CV.OBJECT_DETECTION)
                        .setTypes(Image.class, DetectedObjects.class)
                        .optFilter("backbone", backbone)
                        .optProgress(new ProgressBar())
                        .build();

        try (ZooModel<Image, DetectedObjects> model = ModelZoo.loadModel(criteria)) {
            try (Predictor<Image, DetectedObjects> predictor = model.newPredictor()) {
                DetectedObjects detection = predictor.predict(img);
                saveBoundingBoxImage(img, detection);
                returndetection; }}}private static void saveBoundingBoxImage(Image img, DetectedObjects detection)
            throws IOException {
        Path outputDir = Paths.get("build/output");
        Files.createDirectories(outputDir);

        // Make image copy with alpha channel because original image was jpg
        Image newImage = img.duplicate(Image.Type.TYPE_INT_ARGB);
        newImage.drawBoundingBoxes(detection);

        Path imagePath = outputDir.resolve("detected-dog_bike_car.png");
        // OpenJDK can't save jpg with alpha channel
        newImage.save(Files.newOutputStream(imagePath), "png");
        logger.info("Detected objects image has been saved in: {}", imagePath); }}Copy the code

The output

The problem record

Static interface method calls are not supported at language level ‘7’

Workaround: The POM configuration file explicitly specifies to use JDK8 for compilation.

Question 2: the Exception in the thread. The “main” ai DJL. Repository. Zoo. ModelNotFoundException: No matching model with specified Input/Output type found.

Mxnet :mxnet-model-zoo dependency package is not introduced in the POM configuration file.

Problem 3: An error occurred after switching to the PyTorch engine.

Switching mode:

<dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-engine</artifactId>
</dependency>
<dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-native-auto</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-model-zoo</artifactId>
</dependency>
Copy the code

Error message:

[2021-01-01 23:49:12.420] [WARN] - [main] AI.djl.engine.Engine - Failed to load engine from: ai.djl.pytorch.engine.PtEngineProvider ai.djl.engine.EngineException: Failed to load PyTorch native library at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:56) ~ [pytorch - engine - 0.9.0. Jar:?] At ai. DJL. Pytorch. Engine. PtEngineProvider. GetEngine (PtEngineProvider. Java: 27) ~ [pytorch - engine - 0.9.0. Jar:?] At ai. DJL. Engine. Engine. InitEngine (engine. Java: 59)/API - 0.9.0. Jar:? At ai. DJL. Engine. Engine. < clinit > (49) engine. Java: / API - 0.9.0. Jar:?  at org.town.ObjectDetection.predict(ObjectDetection.java:45) [classes/:?]  at org.town.ObjectDetection.main(ObjectDetection.java:36) [classes/:?] Under Caused by: Java. Lang. UnsatisfiedLinkError: C: \ Users \ steel \. DJL. Ai \ pytorch \ 1.7.0 - CPU - win - x86_64 \ asmjit DLL: Can't find dependent libraries at java.lang.classloader $NativeLibrary. Load (Native Method) ~[?:1.8.0_251] at Java. Lang. This. LoadLibrary0 (1934). This Java: ~ [? : 1.8.0 comes with _251] the at Java. Lang. This. LoadLibrary (1817). This Java: ~ [? : 1.8.0 comes with _251] at Java lang. Runtime. Load0 (809). The Runtime Java: ~ [? : 1.8.0 comes with _251] at Java lang. System. The load (System. Java: 1086) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ForEachOps $ForEachOp $OfRef. Accept (ForEachOps. Java: 184) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ReferencePipeline $3 $1. Accept (ReferencePipeline. Java: 193) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ReferencePipeline $2 $1. Accept (ReferencePipeline. Java: 175) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ReferencePipeline $3 $1. Accept (ReferencePipeline. Java: 193) ~ [? : 1.8.0 comes with _251] the at Java. Util. Iterator. ForEachRemaining (Iterator. Java: 116) ~ [? : 1.8.0 comes with _251] the at Java. Util. Spliterators $IteratorSpliterator. ForEachRemaining (Spliterators. Java: 1801) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. AbstractPipeline. CopyInto (AbstractPipeline. Java: 482) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. AbstractPipeline. WrapAndCopyInto (AbstractPipeline. Java: 472) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ForEachOps $ForEachOp. EvaluateSequential (ForEachOps. Java: 151) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ForEachOps $ForEachOp $OfRef. EvaluateSequential (ForEachOps. Java: 174) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. AbstractPipeline. Evaluate (AbstractPipeline. Java: 234) ~ [? : 1.8.0 comes with _251] the at Java. Util. Stream. ReferencePipeline. ForEach (ReferencePipeline. Java: 418) ~ [? : 1.8.0 comes with _251] the at Ai. DJL. Pytorch. Jni. LibUtils. LoadWinDependencies (LibUtils. Java: 119) ~ [pytorch - engine - 0.9.0. Jar:?] At ai. DJL. Pytorch. Jni. LibUtils. LoadLibrary (LibUtils. Java: 75) ~ [pytorch - engine - 0.9.0. Jar:?] At ai. DJL. Pytorch. Engine. PtEngine. NewInstance (PtEngine. Java: 44) ~ [pytorch - engine - 0.9.0. Jar:?] . 5 more Exception in thread "main" ai.djl.engine.EngineException: No deep learning engine found. Please refer to https://github.com/awslabs/djl/blob/master/docs/development/troubleshooting.md for more details. at ai.djl.engine.Engine.getInstance(Engine.java:119) at org.town.ObjectDetection.predict(ObjectDetection.java:45) at org.town.ObjectDetection.main(ObjectDetection.java:36) Caused by: ai.djl.engine.EngineException: Failed to load PyTorch native library at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:56) at ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:27) at ai.djl.engine.Engine.initEngine(Engine.java:59) at ai.djl.engine.Engine.<clinit>(Engine.java:49) ... 2 more under Caused by: Java. Lang. UnsatisfiedLinkError: C: \ Users \ steel \. DJL. Ai \ pytorch \ 1.7.0 - CPU - win - x86_64 \ asmjit DLL: Can't find dependent libraries at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817) at  java.lang.Runtime.load0(Runtime.java:809) at java.lang.System.load(System.java:1086) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.Iterator.forEachRemaining(Iterator.java:116) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at ai.djl.pytorch.jni.LibUtils.loadWinDependencies(LibUtils.java:119) at ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:75) at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:44) ... 5 moreCopy the code

Solution: not found.

An overview of the

Engineering structures,

Creating a New Maven Project

The project structure

Depend on the introduction of

Log configuration file

Business code

The output

The problem record

Related Posts

Popular science Nginx and Apache differences and advantages and disadvantages comparison

Machine learning/deep learning materials

Deepin-wine-for-ubuntu Install common software Ubuntu