Artwork

Nội dung được cung cấp bởi Real Python. Tất cả nội dung podcast bao gồm các tập, đồ họa và mô tả podcast đều được Real Python hoặc đối tác nền tảng podcast của họ tải lên và cung cấp trực tiếp. Nếu bạn cho rằng ai đó đang sử dụng tác phẩm có bản quyền của bạn mà không có sự cho phép của bạn, bạn có thể làm theo quy trình được nêu ở đây https://vi.player.fm/legal.
Player FM - Ứng dụng Podcast
Chuyển sang chế độ ngoại tuyến với ứng dụng Player FM !

Scaling Data Science and Machine Learning Infrastructure Like Netflix

59:17
 
Chia sẻ
 

Manage episode 293049492 series 2637014
Nội dung được cung cấp bởi Real Python. Tất cả nội dung podcast bao gồm các tập, đồ họa và mô tả podcast đều được Real Python hoặc đối tác nền tảng podcast của họ tải lên và cung cấp trực tiếp. Nếu bạn cho rằng ai đó đang sử dụng tác phẩm có bản quyền của bạn mà không có sự cho phép của bạn, bạn có thể làm theo quy trình được nêu ở đây https://vi.player.fm/legal.

Would you move your data science project from a laptop to the cloud? Would you also like to have snapshots of your project saved along the way so that you can go back in time or share the state of your project with another team member? This week on the show, we have Savin Goyal from Netflix. Savin is the technical lead for machine learning infrastructure at Netflix. He joins us to talk about Metaflow, an open-source tool to simplify building, managing, and scaling data science projects.

Metaflow addresses the needs of the numerous data scientists who work at Netflix. Machine learning is key strength for the streaming service. They tried several existing tools to scale their own internal infrastructure and after this experimentation developed Metaflow.

We talk about the history of the project and how someone could get started with the open-source version. Savin also contrasts the cost of infrastructure as compared to data scientists and the cost of their time.

Course Spotlight: Simplify Python GUI Development With PySimpleGUI

In this step-by-step course, you’ll learn how to create a cross-platform graphical user interface (GUI) using Python and PySimpleGUI. A graphical user interface is an application that has buttons, windows, and lots of other elements that the user can use to interact with your application.

Topics:

  • 00:00:00 – Introduction
  • 00:01:53 – What is Metaflow?
  • 00:04:15 – Savin’s background in data science and infrastructure
  • 00:06:06 – Democratization of infrastructure and iteration of tools
  • 00:10:34 – What information is saved about the infrastructure requirements for a project?
  • 00:17:17 – How are the requirements annotated?
  • 00:18:39 – Sponsor: Digital Ocean’s App Platform
  • 00:19:15 – How do project snapshots work?
  • 00:29:33 – Cost of infrastructure vs data scientists
  • 00:32:28 – Working with data at Netflix scale
  • 00:37:55 – Video Course Spotlight
  • 00:39:06 – Getting an organization to use new tools and then making open-source
  • 00:49:51 – Documentation of Metaflow and getting started on solving infrastructure problems
  • 00:53:57 – What made you interested in working on infrastructure tools?
  • 00:55:13 – What is something you are excited about in the world of Python?
  • 00:56:18 – What do you want to learn next?
  • 00:58:14 – Thanks and goodbye

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

  continue reading

198 tập

Artwork
iconChia sẻ
 
Manage episode 293049492 series 2637014
Nội dung được cung cấp bởi Real Python. Tất cả nội dung podcast bao gồm các tập, đồ họa và mô tả podcast đều được Real Python hoặc đối tác nền tảng podcast của họ tải lên và cung cấp trực tiếp. Nếu bạn cho rằng ai đó đang sử dụng tác phẩm có bản quyền của bạn mà không có sự cho phép của bạn, bạn có thể làm theo quy trình được nêu ở đây https://vi.player.fm/legal.

Would you move your data science project from a laptop to the cloud? Would you also like to have snapshots of your project saved along the way so that you can go back in time or share the state of your project with another team member? This week on the show, we have Savin Goyal from Netflix. Savin is the technical lead for machine learning infrastructure at Netflix. He joins us to talk about Metaflow, an open-source tool to simplify building, managing, and scaling data science projects.

Metaflow addresses the needs of the numerous data scientists who work at Netflix. Machine learning is key strength for the streaming service. They tried several existing tools to scale their own internal infrastructure and after this experimentation developed Metaflow.

We talk about the history of the project and how someone could get started with the open-source version. Savin also contrasts the cost of infrastructure as compared to data scientists and the cost of their time.

Course Spotlight: Simplify Python GUI Development With PySimpleGUI

In this step-by-step course, you’ll learn how to create a cross-platform graphical user interface (GUI) using Python and PySimpleGUI. A graphical user interface is an application that has buttons, windows, and lots of other elements that the user can use to interact with your application.

Topics:

  • 00:00:00 – Introduction
  • 00:01:53 – What is Metaflow?
  • 00:04:15 – Savin’s background in data science and infrastructure
  • 00:06:06 – Democratization of infrastructure and iteration of tools
  • 00:10:34 – What information is saved about the infrastructure requirements for a project?
  • 00:17:17 – How are the requirements annotated?
  • 00:18:39 – Sponsor: Digital Ocean’s App Platform
  • 00:19:15 – How do project snapshots work?
  • 00:29:33 – Cost of infrastructure vs data scientists
  • 00:32:28 – Working with data at Netflix scale
  • 00:37:55 – Video Course Spotlight
  • 00:39:06 – Getting an organization to use new tools and then making open-source
  • 00:49:51 – Documentation of Metaflow and getting started on solving infrastructure problems
  • 00:53:57 – What made you interested in working on infrastructure tools?
  • 00:55:13 – What is something you are excited about in the world of Python?
  • 00:56:18 – What do you want to learn next?
  • 00:58:14 – Thanks and goodbye

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

  continue reading

198 tập

Tất cả các tập

×
 
Loading …

Chào mừng bạn đến với Player FM!

Player FM đang quét trang web để tìm các podcast chất lượng cao cho bạn thưởng thức ngay bây giờ. Đây là ứng dụng podcast tốt nhất và hoạt động trên Android, iPhone và web. Đăng ký để đồng bộ các theo dõi trên tất cả thiết bị.

 

Hướng dẫn sử dụng nhanh