From Model Zoo, the heart of the machine.

Researchers and developers have been searching the GitHub repository for interesting projects and implementations for a long time, but we might need a specific topic and do some filtering. A developer recently launched a model aggregation platform where we can search for the most popular open source models by field. The authors classify these models according to GitHub project collection, project domain, implementation framework and other characteristics, so that we can quickly locate the appropriate model.

Project Address: Modelzoo.co /

Deep learning is the most popular subfield of machine learning these days, with its powerful modeling capabilities for high-dimensional data, and it shines in many different tasks and fields, such as computer vision, natural language processing, and agent learning. A deep learning model is a representation learning approach that learns the representation itself from the model, rather than just mapping the representation to the output as traditional machine learning does.

At present, deep learning mainly extracts features from raw data automatically through neurons at different levels, and the representations it learns often have better performance than traditional ML manually designed representations. Currently, most of the ML implementations on GitHub are deep learning models, which are built to solve specific problems one by one in different hierarchies and network architectures. For example, 2D convolution layers can be stacked in the form of layers to extract features of TWO-DIMENSIONAL data (such as images) from simple to complex, while the stacking of cyclic units is suitable for processing sequence labeling problems such as machine translation. This, coupled with modules such as residual connections, gating mechanisms and attention mechanisms, enables task-specific learning.

These deep models of hierarchical representation have been studied extensively in recent years, with many developers and researchers sharing their code and ideas. ModelZoo aims to provide a centralized platform for searching good deep models, and to provide resources for reproducing papers, building complex neural networks, learning model code, and working directly with pre-trained models.

Model Zoo is a newly built platform, so the library of models it contains will continue to evolve. If readers find other good models that have not been included, they can submit them on the website and notify the authors. In addition, the developer of the website, currently an undergraduate student in Singapore, hopes to use such a platform to help more learners.

All models collected by Model Zoo can be found in GitHub. Author Jing Yu sorted out models according to GitHub project collection size, project domain, implementation framework and other characteristics. In addition, we can also use keyword search to determine the models to be searched. Each model has a README document for the GitHub project displayed on the page, so readers can quickly determine the basic information and requirements of the project. In addition, the author also builds a Newsletter module, which sorts out the most popular and important models of the month once a month.

Currently, the deep learning models in the project fall into five categories: computer vision, natural language processing, generative models, reinforcement learning and unsupervised learning. These categories or fields are overlapping with each other. For example, many generative models can be classified as unsupervised learning, and in fact many models can be classified as multiple categories.

At present, we include Tensorflow, Caffe, Caffe2, PyTorch, MXNet and Keras. Although many frameworks are not included, such as the original implementation of CycleGAN because of its Lua language and Torch framework, even though the project has a collection of 6453, this platform is also not included.

You can enter the above framework classification interface and model classification interface respectively from Browse Frameworks and Browse Categories, and Browse according to their Categories. The search box below supports searching for model names.

Models are sorted by GitHub collection (Magenta’s 10164 STAR, for example), and the preview screen also contains project profiles, implementation frameworks, and categories. Click to go to the project details (that is, the README document corresponding to the GitHub project). Here’s a quick look at some of the most popular projects in each area:


Computer vision

  • Making star: 10164
  • Implementation framework: TensorFlow

Magenta is a research project exploring the role of machine learning in art and music creation, which has been divided into the fields of computer vision and reinforcement learning by various authors. It mainly involves developing new deep learning and reinforcement learning algorithms for generating songs, images, sketches, etc. But it is also trying to build smart tools and interfaces to help artists and musicians expand the way they use these models. Magenta was started by some researchers and engineers from Google’s Brain team, but many others made important contributions to the project.

Currently, the project has very more application implementation and demonstrate, readers can see magenta.tensorflow.org/blog.

Mask-RCNN

  • Making star: 6104
  • Implementation framework: Keras

Mask R-CNN is a two-stage framework. The first stage scans the image and generates candidate regions (the regions that might contain a target), and the second stage classifies candidate regions and generates boundary boxes and masks. Mask R-CNN is extended from Faster R-CNN, which was proposed by the same author last year. Faster R-CNN is a popular target detection framework, and Mask R-CNN extends it to instance segmentation framework.

The project is an implementation of Mask r-cnn on Python3, Keras, and TensorFlow. The model can generate borders and segmentation masks for target instances in images. Its architecture is based on the feature pyramid network (FPN) and ResNet101 backbone network.

The REPo contains: source code for Mask R-CNN built on FPN and ResNet101; Data set MS COCO training code; Pre-training weights for MS COCO; Visualize the Jupyter Notebook for each step of the inspection process; ParallelModel classes for multi-GPU training; Evaluation of MS COCO index (AP); Examples of training custom data sets.


Natural language processing

WaveNet

  • Making star: 3692
  • Implementation framework: TensorFlow

The main structure of WaveNet is shown in the figure below. It is a fully convolutional neural network, in which the convolutional layer has different dilation factors, which makes its receptive field grow exponentially according to the hierarchical depth and can cover thousands of time steps. In addition, Wave Net was the first to propose a causal convolution model. Intuitively, it is similar to “splitting” the convolution operation in half so that it can only operate on the input of past time steps. In fact, many subsequent models using convolutional networks to deal with sequence annotation problems refer to the two structures proposed by WaveNet.

WaveNet neural network architecture proposed by DeepMind can directly generate new audio waveforms, which has superior performance in text-to-speech conversion and audio generation. During training, the input sequence was a real waveform recorded by a human speaker. After training, we can sample the network to generate synthetic speech. At each time step of sampling, a value is taken from the probability distribution calculated by the network. This value is then fed back into the input and generates a new prediction for the next step. Building samples one step at a time like this is computationally expensive, but it’s critical to producing complex, reality-sounding audio.


Sentence Classification with CNN

  • Making star: 3416
  • Implementation framework: TensorFlow

Convolutional Neural Networks for Sentence Classification is a simplified TensorFlow implementation.

Generate models

DCGAN-tensorflow

  • Making star: 4263
  • Implementation framework: TensorFlow

GAN proposed by Goodfellow et al is a new framework for generating models through adversarial process estimation. Under this framework, we need to train two models simultaneously, namely, a generation model G that can capture data distribution and a discriminant model D that can estimate the probability of data coming from real samples. The training process for generator G is to maximize the probability that the discriminator will make an error, that is, the discriminator will mistakenly believe that the data is a real sample rather than a fake sample generated by the generator. Thus, this framework corresponds to a minimax game with two players. DCGAN mainly uses convolutional neural network as the architecture of generator G and discriminator D, so it can generate relatively clear and excellent images.

TensorFlow implementation of Deep Convolutional Generative Adversarial Networks is a stable Generative Adversarial network.

The diagram above shows the architecture of generator G, which uses four transposed convolution for upsampling, restoring a 100-dimensional random variable to the image. In order to prevent the discriminator network from converging too fast, the generator network will be continuously iterated twice when the discriminator iterates once, which is different from the original paper.


Image-to-Image Translation with Conditional Adversarial Networks

  • Making star: 4242
  • Implementation framework: PyTorch

Because the original implementation of CycleGAN was written in Lua, many researchers use the project to reproduce image-to-image transformations. CycleGAN’s main idea is to train two pairs of generator-discriminator models to transform images from one domain to another. In this process we require cyclic consistency, that is, after applying the generator to the image, we should get an image similar to the original L1 loss. Therefore, we need a cyclic loss function that ensures that the generator does not convert an image from one domain to another that is completely unrelated to the original.

This project is a PyTorch implementation of paired and unpaired image-to-image conversion.


Reinforcement learning

Deep Reinforcement Learning for Keras

  • Making star: 2639
  • Implementation framework: Keras

Keras-rl makes it easy to use the best current deep reinforcement learning algorithms, implemented using KerAS and combined with OpenAI Gym to build projects.

At present, the project has implemented the following algorithms:

  • Deep Q Learning (DQN)
  • Double DQN
  • Deep Deterministic Policy Gradient (DDPG)
  • Continuous DQN (CDQN or NAF)
  • Cross-Entropy Method (CEM)
  • Dueling network DQN (Dueling DQN)
  • Deep SARSA


DQN-tensorflow

  • Making star: 1610
  • Implementation framework: TensorFlow

The realization of TensorFlow in Human-level Control through Deep Reinforcement Learning.

The implementation includes: Deep Q-network and Q-learning

  • Experience replay memory
  • Reduce the relevance of continuous updates
  • The network for the q-learning target has been modified due to interval problems
  • Reduce the association between the target and the predicted Q value


Unsupervised learning

MUSE: Multilingual Unsupervised and Supervised Embeddings

  • Making star: 1298
  • Implementation framework: PyTorch

MUSE is a Python library for multilingual word embedding that aims to provide the community with the best current fastText-based multilingual word embedding, as well as large-scale, high-quality bilingual dictionaries for training and evaluation.

The project used two methods, one supervised using a bilingual dictionary or the same string, and the other unsupervised without using any parallel data.


Domain Transfer Network (DTN)

  • Making star: 654
  • Implementation framework: TensorFlow

The TensorFlow implementation of Unsupervised cross-domain Image Generation is shown in the following figure.

The above is a brief introduction of the main popular models in each category, and there are many excellent projects that are not shown. Readers should select models according to the platform and specific needs.