Thread by @MLinProduction, I asked data scientists what challenges they were facing in 2020 and [...]

I asked data scientists what challenges they were facing in 2020 and the resounding answer was how difficult it was to deploy models to production. I thought writing 3-4 blog posts would solve it.

I& #39;m 8 posts in and still so much to say. Here& #39;s a breakdown of each post

https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Down pointing backhand index" aria-label="Emoji: Down pointing backhand index">

Part 1 - What does it even mean to "deploy a model?" How does deployment fit into the machine learning process? What factors should you take into consideration when deciding how to deploy? https://mlinproduction.com/what-does-it-mean-to-deploy-a-machine-learning-model-deployment-series-01/">https://mlinproduction.com/what-does...

What Does it Mean to Deploy a Machine Learning Model? (Deployment Series: Guide 01) - ML in...

Thinking about deployment as a software engineer rather than as a data scientist will dramatically simplify what it means to deploy a model. Learn more now.

https://mlinproduction.com/what-does-it-mean-to-deploy-a-machine-learning-model-deployment-series-01/

Part 2 - Deployment is considerably easier when you& #39;re working with the right interfaces. Doubly important when you& #39;re using models across different frameworks and languages. So what& #39;s the right interface to make deployment easier? https://mlinproduction.com/software-interfaces-for-machine-learning-deployment-deployment-series-02/">https://mlinproduction.com/software-...

Software Interfaces for Machine Learning Deployment (Deployment Series: Guide 02) - ML in Production

In terms of ML deployment, well constructed interfaces facilitate reproducible, automated, plug-and-play deployments. Learn more from Guide #02 of our deployment series.

https://mlinproduction.com/software-interfaces-for-machine-learning-deployment-deployment-series-02/

Part 3 - If you can precompute and cache predictions in batch, DO IT! It& #39;s much easier than deploying and maintaining APIs and other near real time infrastructure. Here& #39;s how to do batch inference. https://mlinproduction.com/batch-inference-for-machine-learning-deployment-deployment-series-03/">https://mlinproduction.com/batch-inf...

Batch Inference for Machine Learning Deployment (Deployment Series: Guide 03) - ML in Production

Batch inference allows us to generate predictions on a batch of samples. Learn more from Guide #03 of our deployment series.

https://mlinproduction.com/batch-inference-for-machine-learning-deployment-deployment-series-03/

Part 4 - But when you need predictions in real time, you need online inference. There are many gotchas in online inference: you need to query data from multiple sources in real time, you& #39;ll need A/B testing, you need rollout strategies... https://mlinproduction.com/the-challenges-of-online-inference-deployment-series-04/">https://mlinproduction.com/the-chall...

The Challenges of Online Inference (Deployment Series: Guide 04) - ML in Production

Deploying machine learning models for online inference is more challenging than deploying models for batch inference. Learn more from our guide.

https://mlinproduction.com/the-challenges-of-online-inference-deployment-series-04/

Part 5 - If after learning about those challenges you decide you still need online inference, bless your heart. There are a lot of posts on Flask APIs, but that& #39;s the easiest part. You need versioning, autoscaling, and the ability to A/B test models. https://mlinproduction.com/online-inference-for-ml-deployment-deployment-series-05/">https://mlinproduction.com/online-in...

Online Inference for ML Deployment (Deployment Series: Guide 05) - ML in Production

In this deployment series post we'll demonstrate how to implement online inference. We'll begin by discussing when online inference is and isn't required.

https://mlinproduction.com/online-inference-for-ml-deployment-deployment-series-05/

Part 6 - Where do you store all these trained models? Where do you track metadata and lineage? How do you retrieve models at inference time? That& #39;s where you& #39;ll need a model registry. https://mlinproduction.com/model-registries-for-ml-deployment-deployment-series-06/">https://mlinproduction.com/model-reg...

Model Registries for ML Deployment (Deployment Series: Guide 06) - ML in Production

In this post we'll discuss what an ML model registry is, how to implement a simple model registry, and introduce a popular open source registry.

https://mlinproduction.com/model-registries-for-ml-deployment-deployment-series-06/

Part 7 - It& #39;s not enough to use aggregate metrics to understand model performance. You need to know how the model does on subslices of data. You need machine learning unit tests. https://mlinproduction.com/testing-machine-learning-models-deployment-series-07/">https://mlinproduction.com/testing-m...

Test-Driven Machine Learning Development (Deployment Series: Guide 07) - ML in Production

We define what it means to test machine learning models and discuss several different ways of testing models using offline tests.

https://mlinproduction.com/testing-machine-learning-models-deployment-series-07/

Part 8 - Just because a model passes its unit tests, doesn& #39;t mean it will move the product metrics. The only way to establish causality is through online validation. Like any other feature, models need to be A/B tested https://mlinproduction.com/ab-test-ml-models-deployment-series-08/">https://mlinproduction.com/ab-test-m...

A/B Testing Machine Learning Models (Deployment Series: Guide 08) - ML in Production

In this post we describe why it's necessary to A/B test machine learning models and discuss an architecture for A/B testing ML models.

https://mlinproduction.com/ab-test-ml-models-deployment-series-08/

I& #39;m planning on writing a few more posts. Next one will be on rollout strategies (dark mode vs canary vs blue green). But if there& #39;s something you think I missed, shout it out!

Latest Threads Unrolled: