Officially released April 21, PyTorch users can use TorchServe (PyTorch Ecosystem’s new Model Service framework) to deploy trained models on a large scale without having to write custom code.
PyTorch is an open source machine learning framework originally created by Facebook and popular among machine learning researchers and data scientists due to its ease of use.
But deploying and managing models in production is often the most difficult part of the machine learning process, requiring customers to write predictive apis and extend them, and Google’s Tensorflow is more industrial. The release of TorchServe began to break Pytorch’s passivity.
With TorchServe, you can easily deploy the PyTorch model on a large scale in a production environment. It provides lightweight services with low latency, so you can deploy models for high-performance reasoning. It provides default handlers for the most common applications, such as target detection and text sorting, so you can deploy the model without having to write custom code. With powerful TorchServe capabilities, including multi-model services, model versioning for A/B testing, metrics for monitoring, and RESTful endpoints for application integration, you can quickly move models from research to production. TorchServe supports any machine learning environment, including Amazon SageMaker, Kubernetes, Amazon EKS, and Amazon EC2.
TorchServe is built and maintained by AWS in partnership with Facebook as part of the PyTorch open source project. The relevant code is available for download on GitHub.