Uber Introduces PyML: Their Secret Weapon for Rapid Machine Learning Development | Towards Data Science

Uber has been one of the most active companies trying to accelerate the implementation of real world machine learning solutions. Just this year, Uber has introduced technologies like Michelangelo, Pyro.ai and Horovod that focus on key building blocks of machine learning solutions in the real world. This week, Uber introduced another piece of its machine learning stack, this time aiming to short the cycle from experimentation to product. PyML, is a library to enable the rapid development of Python applications in a way that is compatible with their production runtime.

The problem PyML attempts to address is one of those omnipresent challenges in large scale machine learning applications. Typically, there is a tangible mismatch between the tools and frameworks used by data scientists to prototype models and the corresponding production runtimes. For instance, its very common for data scientists to use Python-based frameworks such as PyTorch or Keras for producing experimental models that then need to be adapted to a runtime such as Apache Spark ML Pipelines that brings very specific constraints. Machine learning technologists refer to this issue as a tradeoff between flexibility and resource-efficiency. In the case of Uber, data scientists were building models in Python machine learning frameworks which needed to be refactored by the Michelangelo team to match the constraints of Apache Spark pipelines.

Read complete article here:

Uber Introduces PyML: Their Secret Weapon for Rapid Machine Learning Development.