Netflix's latest release is an open source data science library
Netflix and AWS are teaming up to release Metaflow, a "human-friendly Python library that helps scientists and engineers build and manage real-life data science projects."
Developed at Netflix, Metaflow helps data scientists by providing a unified API to the infrastructure stack that is needed to complete data science projects, from prototype to production.
The impact? Less engineering and more data science. Data scientists currently juggle many different parts of the development stack in any data science project—including versioning, architecture, data warehouses, and more. Keeping track of data and models is time-consuming and complicated.
Metaflow abstracts away some of the engineering complexity of these data science workflows.
Metaflow also pulls data science farther into the AWS sphere of influence. Metaflow works with both local and remote environments, but when data scientists need more data or computing power, they can easily move from local to remote modes of code execution. Netflix has optimized Metaflow to work directly with AWS.
What about the developers? Where data goes, developers will follow.
As more engineering teams become data-driven, AWS will need to convince teams of data scientists and developers to keep their data on the AWS cloud.
With more automation that saves data scientists and developers from tedious setup and maintenance, AWS plans to make serious inroads in hooking data-focused teams.
Want to get more of these in your inbox?
Subscribe for weekly updates from the Software team.