Deeplearning4j aims to integrate models with Java applications, providing a set of components for building JVM-based applications that include AI

Eclipse Deeplearning4j is an open source distributed deep learning library for JVMS. Deeplearning4j is written in Java and is compatible with any JVM language, such as Scala, Clojure, or Kotlin. The underlying calculations are written in C, C++, and Cuda. Keras will be used as a Python API. Deeplearning4j integrates with Hadoop and Apache Spark to bring AI into the business environment for use on distributed Gpus and cpus.

Deeplearning4j is actually a bunch of projects designed to support all the requirements of a JVM-based deep learning application. In addition to Deeplearning4j itself (advanced API), it includes ND4J (general linear algebra), SameDiff (automatic plotting based differentiation), DataVec (ETL), Arbiter (hyperparameter search), and C++ LibND4J (supports all of the above). LibND4J in turn calls standard libraries to support cpus and Gpus, such as OpenBLAS, OneDNN (McL-dnn), cuDNN, and cuBLAS.

The goal of Eclipse Deeplearning4j is to provide a core set of components for building applications that include AI. AI products within the enterprise often have a broader scope than machine learning. The overall goal of this release is to provide intelligent defaults for building deep learning applications.

Deeplearning4j competes with all other deep learning frameworks to some extent. The most comparable project in the range is TensorFlow, the leading end-to-end deep learning production framework. TensorFlow currently has Python, C++, and Java (experimental) interfaces, as well as a separate implementation of JavaScript. TensorFlow uses two training methods: graph-based and eager execution. Deeplearning4j currently supports graph-based execution only.

PyTorch, perhaps the leading deep learning research framework, supports only real-time modes; It has Python, C++, and Java interfaces. H2O Sparkling Water integrates the H2O open source distributed memory machine learning platform with Spark. H2O has interfaces for Java and Scala, Python, R, and H2O Flow notebooks.

Commercial support for Deeplearning4j can be purchased from Konduit, which also supports many developers working on the project.

How Deeplearning4j works

Deeplearning4j treats the task of loading the data and training the algorithm as separate processes. You use the DataVec library to load and transform data, and use the tensors and ND4J libraries to train models.

You through a RecordReader interface data, and USES RecordReaderDataSetIterator. You can choose a DataNormalization class as your DataSetIterator. Using ImagePreProcessingScaler image data, NormalizerMinMaxScaler if the uniform range of all sizes along the input data, NormalizerMartial Arts for most other cases. If necessary, you can implement custom DataNormalization classes.

The DataSet object is a container for data characteristics and labels, and stores the values in INDArray instances: one for sample characteristics, one for labels, and two for masking (if you are using time series data). In the case of Features, INDArray is a tensor of size Number of Examples x Number of Features. Typically, you break your data into small batches for training; The number of examples in an INDArray is small enough to fit into memory, but large enough to get a good gradient.

If you look at the Deeplearning4j code used to define the model, such as the Java example below, you will see that it is a very high-level API, similar to Keras. In fact, Deeplearning4j’s planned Python interface will use Keras; Now, if you have a Keras model, you can import it into Deeplearning4j.

MultiLayerConfiguration conf = New NeuralNetConfiguration. Builder (). OptimizationAlgo (OptimizationAlgorithm. STOCHASTIC_GRADIENT_DESCENT). Update the program (new Nesterovs (learningRate,0.9)). List (new DenseLayer. Generator (). N in (numInputs). NOUT (numHiddenNodes). Active (" RELU "). Build (), a new output layer. Generator (LossFunction. NEGATIVELOGLIKELIHOOD). Activation (" Softmax "). N In (numHiddenNodes). NOut (Num-state). Build ()). Back propagation (true). Construct ();Copy the code

The MultiLayerNetwork class is available in Eclipse Deeplearning4j’s simplest network configuration API; For DAG structures, ask ComputationGraph to use them instead. Note that the optimization algorithm (SGD in this case) is specified separately from the updater (Nesterov in this case). This very simple neural network has a dense layer with ReLU activation function and an output layer with -log(likelihood) loss and Softmax activation function and is solved by back propagation. More complex networks may also have GravesLSTM, ConvolutionLayer, EmbeddingLayer, and other supported two-dozen layer types and 16-layer space types.

The simplest way to train the model is for.fit() to call the methods on the model configuration with arguments to your DataSetIterator. You can also reset the iterator and call the method over any.fit() period as needed, or use the EarlyStoppingTrainer.

To test the model performance, use a Evaluation class to see how well the training model fits the test data, which should not be the same as the training data.

Deeplearning4j provides a listener tool that helps you visually monitor the performance of your network, which is invoked after each small batch. One of the most common listeners is ScoreIterationListener.

Install and test Deeplearning4j

Currently, the easiest way to try out Deeplearning4j is to use the official Quick Start. It requires a relatively new Java version, an installation of Maven, a working Git, and a copy of IntelliJ IDEA (preferred) or Eclipse. There are also some user-provided quick starts. Start by cloning the Eclipse/Deeplearning4J-Examples repository onto your own machine using Git or GitHub Desktop. Then use Maven to install the project from the DL4J-examples folder.

Martinheller @Martins - Retina - MacBook DL4J - Instances % MVN New Install [INFO] Scan project... [WARNING] [WARNING] part of the problem, while establishing an effective pattern for organization. Deeplearning4j: dl4j - Instance: jar:1.0 。0-beta7 [warning]'build.plugins.plugin.(groupId:artifactId)'Must be unique, but duplicate declaration of plug-in org found. Apache. A connoisseur. Plugins: connoisseur - compile - plugins @ line250Column,21[WARNING] [WARNING] This is highly recommended because they threaten the stability of your build, resolve these issues. [WARNING] [WARNING] For this reason, future versions of Maven may not support the creation of such malformed projects. [Warning] [info] [info] ------------------< organization. Deeplearning4j: dl4j - examples > -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- [information] dl4j1.0Introduction to build.0- beta7 [INFO] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- (jar) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- to download from the central: HTTPS:/ / repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-enforcer-plugin/1.0.1/maven-enforcer-plugin-1.0.1.pom from centralDownload: HTTPS:/ / repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-enforcer-plugin/1.0.1/maven-enforcer-plugin-1.0.1.pom (6.5 KB at 4.4kb /s) / / repo.maven.apache.org/maven2/org/apache/maven/enforcer/enforcer/1.0.1/enforcer-1.0.1.pom downloaded from central: HTTPS:
 
 / / repo.maven.apache.org/maven2/org/apache/maven/enforcer/enforcer/1.0.1/enforcer-1.0.1.pom kB at 137 kB/s (11) from the centerDownload: HTTPS:// Repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-enforcer-plugin/1.0.1/maven-enforcer-plugin-1.0.1.jar downloaded from central: HTTPS: / / repo. Maven. Apache.org/maven2/org/apache/maven/plugins/maven-enforcer-plugin/1.0.1/maven-enforcer-plugin-1.0.1.jar (22 KB at 396 kB/s) / / repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.4.0/exec-maven-plugin-1.4.0.pomDownload from the central: HTTPS:/ / repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.4.0/exec-maven-plugin-1.4.0.pom (12 kB at 283 KB /s) from the central download: HTTPS: / / repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.4.0/exec-maven-plugin-1.4.0.jar downloaded from central: HTTPS: / / repo.maven.apache.org/maven2/org/codehaus/mojo/exec-maven-plugin/1.4.0/exec-maven-plugin-1.4.0.jar (46 kB at 924 kB /s) download from central: HTTPS:
 
 
 / / repo.maven.apache.org/maven2/com/lewisd/lint-maven-plugin/0.0.11/lint-maven-plugin-0.0.11.pom from centralDownload: HTTPS:/ / repo. Maven. Apache. Org/maven2 / com/lewisd/lint - maven plugin / 0.0.11 / lint - maven plugin - 0.0.11. Pom (19 kB at 430 KB/s) from the central download: HTTPS: / / repo. Maven. Apache.org/maven2/com/lewisd/lint-maven-plugin/0.0.11/lint-maven-plugin-0.0.11.jar downloaded from central: HTTPS: / / repo.maven.apache.org/maven2/ com/lewisd/lint - maven plugin / 0.0.11 / lint - maven plugin - 0.0.11. Jar (106 kB at 1.6 MB/s) from the central download
 
 
 : https : / / repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.5.1/maven-compiler-plugin-3.5.1.pom. [Warning] - org.agrona. The collection. Hash [warning] -org. Agrona. The collection. Long2ObjectCache$ValueIterator [warning] - Organization. Agrona. The collection. Int2ObjectHashMap$EntrySet [warning] - Organization. Agrona. Concurrency. SleepingIdleStrategy [warning] - org. Agrona. The collection. MutableInteger [warning] -org. Agrona. The collection. Int2IntHashMap [warning] -org. Agrona. The collection. IntIntConsumer [Warning] - Organization. Agrona. Simultaneous.state. StatusIndicator -175More... [Warning] Javafx-base -14- the MAC. Jar, javafX-graphics -14- the MAC. Jar, Jakarta. XML. Binding - API -2.3 。2.The jar is defined1Three overlapping classes: [WARNING] - module - Protobuf - for information [WARNING]1.0 。0- beta.7. The jar, the guava19.0. Can limit3Overlapping classes: [WARNING] -com. Google. A third party. Common suffix. TrieParser [Warning] -com. Google. A third party. Common suffix. PublicSuffixPatterns [Warning] - com Google. A third party. Common suffix. PublicSuffixType [ WARNING ] JSR305 -3.0 。2.Jar, guava -1.0 。0- beta.7. Can limit35Overlapping classes: [WARNING] - javax. The comments. Regular expression [warning] - javax. comment. Concurrency. Immutable [warning] - Javax. The comments. Yuan. TypeQualifierDefault [Warning] - Javax. The comments. Yuan. TypeQualifier [Warning] - Javax. The comments. Syntax [warning] - javax. The comments. Check the return value [warning] -javax. The comments. CheckForNull [warning] - javax. The comments. Non-empty [warning] - javax. The comments. Yuan. TypeQualifierNickname [Warning] - javax. The comments. MatchesPattern [warning] - There's more25A... [Warning] Maven-shade-plugin has detected some class files [warning] in two or more JARS. When this happens, only one [WARNING] single version class is copied to the super jar. [warning] Usually this is not harmful, and you can skip these warnings, [warning], otherwise try to override false [warning] MVN dependencies based on manual: tree-ddetail =trueAnd the output above. [warning] see HTTP:/ / maven.apache.org/plugins/maven-shade-plugin/ INFO. Attach shadow artifact. [INFO] [INFO] -- Maven - install - plugins: 2.4: Install (default - install) @Dl4j - Instance - [INFO] Install/volume/data/Buyback/deeplearning4J - instance/dl4j - instance/target/DL4j - instance -1.0 。0- beta.7. Jar to/Users/martinheller /. M2 / repository/organization/deeplearning4j/dl4j - instance /1.0 。0Beta7/ dl4j - instance -1.0 。0- beta.7. Jar [info] install/volume/data/repos/deeplearning4j-examples/dl4j-examples/POm. XML to/Users/martinheller /. M2 / repository/organization/deeplearning4j/dl4j - example /1.0 . 0Beta7/ dl4j - instance -1.0 。0- beta.7. POM [INFO] Install/volume/data/Repo/Deeplearning4J - instance/DL4j - instance/target/DL4j - instance -1.0 。0- Beta7 - Shadow. Jar to/Users/martinheller /. m2 / repository / org / deeplearning4j / dl4j - examples /1.0 。0Beta7/ dl4j - instance -1.0 。0- beta.7Of - shadow. Jar [information] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- [information] building a successful [information] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- [INFO] total time:05 : 07Min [INFO] Finish time:2020 - 07 - 10T10 : 58 : 55 - 04 : 00[message] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- martinheller @ Martins - Retina - MacBook Dl4j - example %Copy the code

Once installed, open the DL4J-examples/directory using IntelliJ IDEA and try running some examples.

The README under DL4J-examples lists all examples and provides a brief explanation. By the way, you can use the IntelliJ IDEA preference to install a new version of the JDK and apply it to your project.

The famous iris dataset has only 150 samples and is usually easy to model, although some of these irises are often misclassified. The model used here is a three-layer dense neural network.

Running the Iris classifier shown above yields a pretty good fit: accuracy, accuracy, recall, and F1 scores are all about 98%. Note that only one test case was misclassified in the obfuscation matrix.

The linear classifier demonstration runs in seconds and generates probability graphs for training and test data sets. The data is generated specifically for linear classification into two classes.

The multi-layer Sensor (MLP) classification model for MNIST handwritten digital datasets yielded approximately 97% accuracy, accuracy, recall, and F1 scores after approximately 14K iterations. This is not as good or fast as the results of convolutional neural networks (e.g. LeNet) on this data set.

Deeplearning4j performance

Benchmarking Java programs can be tricky. In particular, you need to warm up your code before timing it to eliminate the overhead of the JIT compiler, and you need to ensure that the JVM has enough RAM configured for the program to benchmark. The Deeplearning4j community maintains a benchmark code repository for a variety of popular models and configurations.

Deeplearning4j is as fast as Caffe in non-trivial image recognition tasks using multiple Gpus, according to the developers. To use multiple machines, you can run Deeplearning4j using Spark

Deep learning for Java applications

Overall, Deeplearning4j is a valuable library for loading and moderating data and performing deep learning on the Java VM. While not quite as mature as TensorFlow or PyTorch, Deeplearning4j will appeal to data scientists who want to integrate deep learning models into Java-based applications. Deeplearning4j has the ability to import Keras models, which will simplify the transition for those who want to convert from TensorFlow.

Deeplearning4j supports creating graphs and then running them, just like TensorFlow 1. It does not support real-time mode training, such as TensorFlow 2 and PyTorch. This is not important for production, but it may make using Deeplearning4j less attractive for research.

Deeplearning4j currently has two distributed training mechanisms, parameter averaging and gradient sharing. The latter is preferable, but it was only added to the library in version 1.0.0-beta3. Deeplearning4j also integrates with Spark, which helps support training on a set of machines. Using Spark doesn’t really help when training on a single server with multiple Gpus.

If this article is helpful to you, welcome to like and forward, but also welcome to talk about their own experience in learning, convenient for everyone to learn and grow together! Read more technical dry articles on Java Architecture.