Recently, the author of “A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem” has opened source the project code of this paper. This article is about how to use deep reinforcement learning for portfolio management, and the performance of the DRL framework proposed is much better than other algorithms. Machine heart has a brief translation of the abstract of the paper, with GitHub implementation.


Links to papers: arxiv.org/abs/1706.10…

GitHub address: github.com/ZhengyaoJia…


A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

 


Abstract: Financial portfolio management is the process of continuously allocating funds to different financial products in order to obtain greater cumulative returns. This paper presents a reinforcement learning framework that does not use financial models to provide deep machine learning solutions to portfolio management problems. The framework includes EIIE (Ensemble of Identical Independent Evaluators) topology, Portfolio Vector Memory, PVM), Online Stochastic Batch Learning (OSBL), and reward functions for instant rewards. In this study, the framework is implemented in three cases: convolutional neural network (CNN), basic cyclic neural Network (RNN) and Long short-term memory Network (LSTM). They, along with a number of recently reviewed or published portfolio selection strategies, underwent three post-test experiments in which cryptocurrency markets traded in 30-minute cycles. Cryptocurrencies are decentralized electronic currencies, the most famous of which is Bitcoin. Three instances of the framework consistently ranked among the top three in all experiments, outperforming other trading algorithms. Despite a 0.25% commission rate in a post-test, the framework was able to at least quadruple revenue within 50 days.


Making implementation

This is the latest implementation version of our paper “A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem”. It also includes some other portfolio management algorithms mentioned by Li and Hoi in the review paper.

  • A deep reinforcement learning framework is the core of the library. The basis of this approach is gradient descent over immediate rewards. You can configure topology, training methods, or input data in a separate JSON file. The training process will be recorded and the user can see the training process through tensorboard.
  • A better hyperparametric optimization requires result summarization and parallel training.
  • For comparison purposes, the library also incorporates a financial model based portfolio management algorithm, which implements OLPS, a toolkit based on Li and Hoi.


Differences between paper versions


Note: This library is part of our main project and is several versions ahead of the implementation of the Arxiv V2 paper.

  • Anyway, the version fixes some technical bugs and makes some improvements to hyperparameters and engineering.
  • The biggest bug in the ArXiv V2 paper is that the test time mentioned in the paper is 30% shorter than the real time.
  • Using new hyperparameters, users can train their models quickly (in less than 30 minutes), according to the Network.
  • A later version of the Lent paper includes all the updates.
  • Anyway, the open source version doesn’t include original build history or internal discussions (including some code comments). Some of the ideas have not yet been realized and may become the basis of our future papers, so they are not published yet.


Platform support


Windows: Python 3.5+; Linux: Python 2.7+/3.5+.


dependency


Install dependencies through PIP install -r requirements.txt

• tensorflow (> = 1.0.0)

• tflearn

Pandas,

•…


The user manual

  •  https://github.com/ZhengyaoJiang/PGPortfolio/blob/master/user_guide.md
  •  https://github.com/ZhengyaoJiang/PGPortfolio/wiki/User-Guide