Recently, the paper a-Fast-RCNN: Hard Positive Generation Via Adversary for Object Detection published by Wang Xiaolong of Carnegie Mellon University (CMU) attracted much attention. In this study, the idea of adversarial learning is applied to the image recognition problem, and the detection network is trained by generating occlusion and deformation image samples from adversarial network, and good results are achieved. This paper has been accepted by CVPR2017.


Link to paper: http://www.cs.cmu.edu/~xiaolonw/papers/CVPR2017_Adversarial_Det.pdf

Github：https://github.com/xiaolonw/adversarial-frcnn


A-fast-rcnn: Hard Positive Generation via Adversary for Object Detection



Abstract


How do you determine if an object detector can handle obscured, angled or distorted images? Our current solution is to use a data-driven strategy to collect a huge dataset of what objects look like under all conditions, and hope that model training will teach the classifier to recognize them as the same object. But can the data set really cover everything? We believe that properties such as classification, shading, and deformation are also consistent with the long tail theory. Some shading and distortion are so rare that they almost never happen, and we want to train models that can handle all situations. In this paper, we propose a new solution. We propose an adversarial network that can generate self-shadowing and deformation examples. The object of confrontation is to generate examples of objects that are difficult for detectors to identify. In our architecture, the original recognizer learns with its counterpart. Experimental results show that compared with fast-RCNN, our method has a 2.3% increase in mAP on VOC07 and a 2.6% increase in mAP on VOC2012 object recognition challenge. We also published the code for this study.



Figure 1: In the paper, we propose the example of using adversarial networks to generate objects with occlusion and deformation that make it difficult for object detectors to classify. As the detector’s performance improves, so does the quality of the images produced by the adversarial network. Through this adversarial strategy, the accuracy of neural network recognition is further improved.


Figure 2: Schematic diagram of ASDN architecture of this method and how to combine with Fast RCNN. Our ASDN network uses input images to add patches to the RoI pooling layer. The ASDN network predicts occlusion/EXTREME speck masks, which are then used to discard eigenvalues and passed to the FAST-RCNN classification tower.



Figure 3 :(a) model pre-training — find the occlusion with the highest difficulty for training the ASDN network. (b) Examples of occlusion masks generated by THE ASDN network, where the black area is occluded when passing through the FRCN pipeline.



Figure 4: Combination architecture of ASDN and ASTN. Occlusion masks are first created, then paths are rotated to produce examples for training.



Table 1: Average accuracy of VOC identification test, FRCN refers to THE FRCN score using our training method.


The Caffe implementation of this study is as follows: A-fast-RCNN: Hard Positive Generation via Adversary for Object Detection


introduce


This implementation is Caffe’s version of A-fast-RCNN. Although our initial implementation in the paper was on Torch. But Caffe’s version is simpler, faster and easier to use. We published code for training a-FAST-RCNN training data using the Adversarial Spatial Dropout Network.


The license


This code is distributed under the MIT License (see License file for details).


reference


If you find this content helpful to your research, you can quote it:

@inproceedings{WangCVPR17afrcnn,
    Author = {Xiaolong Wang and Abhinav Shrivastava and Abhinav Gupta},
    Title = {A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection},
    Booktitle = {Conference on Computer Vision and Pattern Recognition ({CVPR})},
    Year = {2017}
}Copy the code


disclaimer


This implementation is built on a fork of the OHEM code, which in turn builds on the Faster R-CNN Python code and Fast R-CNN. Please select the appropriate research paper to cite when using.


OHEM：https://github.com/abhi2610/ohem

Faster – R – CNN Python:https://github.com/rbgirshick/py-faster-rcnn

Fast R-CNN：https://github.com/rbgirshick/fast-rcnn


The results of



Note: The results recorded in the study are based on the VGG16 network.


The installation


Follow the VOC data download and installation specification, as with Faster R-CNN Python.


use


To run the code, type:

./train.sh

Copy the code

It consists of three stages of training:

./experiments/scripts/fast_rcnn_std.sh [GPU_ID] VGG16 pascal_voc
Copy the code

This has been used to train the standard FAST-RCNN 10,000 iterations, and you may need to download models and logs.


Model: http://suo.im/2cgwYG

Log: http://suo.im/39gkhf

./experiments/scripts/fast_rcnn_adv_pretrain.sh [GPU_ID] VGG16 pascal_voc
Copy the code

During the pre-training phase of anti-network, it may be necessary to download models and logs:


Model: http://suo.im/2cgwYG

Log: http://suo.im/1TSiRh

./copy_model.h

Copy the code

The weights used to copy the above two models are used to initialize the union model.

./experiments/scripts/fast_rcnn_adv.sh [GPU_ID] VGG16 pascal_voc
Copy the code

In this step you may need to download the model and log:

Model: http://suo.im/25uFFX

Log: http://suo.im/2UTbnC

The heart of the Machine has the right to pursue legal action against those who violate laws and regulations.

  1. Please note in bold at the beginning of the article: this article is reprinted from the heart of the machine, identify the author, and attach a link to this article.
  2. Please note in bold at the beginning of the wechat official account: This article is reprinted from the heart of machine, mark the author, and set the link to read the original text as the link of this article, please attach the link to the heart of machine synced.
  3. After the website and wechat reprint the specific article, please send a letter to [email protected] for explanation, and note the title of the reprint article, the name of the reprint wechat and the date of reprint.
  4. Machine heart only accept the above reprint methods, the rest are regarded as infringement, if you need business cooperation, please write to [email protected].