DMTK vs MLflow: What are the differences?

What is DMTK? Microsoft Distributed Machine Learning Tookit. DMTK provides a parameter server based framework for training machine learning models on big data with numbers of machines. It is currently a standard C++ library and provides a series of friendly programming interfaces.

What is MLflow? An open source machine learning platform. MLflow is an open source platform for managing the end-to-end machine learning lifecycle.

DMTK and MLflow can be categorized as "Machine Learning" tools.

Some of the features offered by DMTK are:

• DMTK Framework: a flexible framework that supports unified interface for data parallelization, hybrid data structure for big model storage, model scheduling for big model training, and automatic pipelining for high training efficiency.
• LightLDA, an extremely fast and scalable topic model algorithm, with a O(1) Gibbs sampler and an efficient distributed implementation.
• Distributed (Multisense) Word Embedding, a distributed version of (multi-sense) word embedding algorithm.

On the other hand, MLflow provides the following key features:

• Track experiments to record and compare parameters and results
• Package ML code in a reusable, reproducible form in order to share with other data scientists or transfer to production
• Manage and deploy models from a variety of ML libraries to a variety of model serving and inference platforms

DMTK and MLflow are both open source tools. It seems that DMTK with 2.69K GitHub stars and 595 forks on GitHub has more adoption than MLflow with 23 GitHub stars and 13 GitHub forks.