Read on Twitter
You might have finished the engine, but there& #39;s still a lot of work to put the entire car together.
A Machine Learning model is just a small piece of the equation.
A lot more needs to happen. Let& #39;s talk about that.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="🧵" title="Thread" aria-label="Emoji: Thread">https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
A Machine Learning model is just a small piece of the equation.
A lot more needs to happen. Let& #39;s talk about that.
For simplicity& #39;s sake, let& #39;s imagine a model that takes a picture of an animal and classifies it among 100 different species.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Input: pre-processed pixels of the image.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Output: a score for each one of the 100 species.
Final answer is the species with the highest score.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Final answer is the species with the highest score.
There& #39;s a lot of work involved in creating a model like this. There& #39;s even more work involved in preparing the data to train it.
But it doesn& #39;t stop there.
The model is just the start, the core, the engine of what will become a fully-fledged car.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
But it doesn& #39;t stop there.
The model is just the start, the core, the engine of what will become a fully-fledged car.
Unfortunately, many companies are hyper-focused on creating these models and forget that productizing them is not just a checkbox in the process.
Reports are pointing out that ~90% of Data Science projects never make it to production!
I& #39;m not surprised.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Reports are pointing out that ~90% of Data Science projects never make it to production!
I& #39;m not surprised.
Our model predicting species is now ready!
— "Good job, everyone!"
— "Oh, wait. Now what? How do we use this thing?"
Let& #39;s take our model into production step by step.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
— "Good job, everyone!"
— "Oh, wait. Now what? How do we use this thing?"
Let& #39;s take our model into production step by step.
First, we need to wrap the model with code that:
1. Pre-processes the input image
2. Translates the output into an appropriate answer
I call this the "extended model." Complexity varies depending on your needs.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
1. Pre-processes the input image
2. Translates the output into an appropriate answer
I call this the "extended model." Complexity varies depending on your needs.
Frequently, processing a single image at a time is not enough, and you need to process batches of pictures (you know, to speed things up a bit.)
Doing this requires a non-trivial amount of work.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Doing this requires a non-trivial amount of work.
Now we need to expose the functionality of the extended model.
Usually, you can do this by creating a wrapper API (REST or RPC) and have client applications use it to communicate with the model.
Loading the model in memory brings some other exciting challenges.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Usually, you can do this by creating a wrapper API (REST or RPC) and have client applications use it to communicate with the model.
Loading the model in memory brings some other exciting challenges.
Of course, we can& #39;t trust what comes into that API, so we need to validate its input:
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">What& #39;s the format of the image we are getting?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">What happens if it doesn& #39;t exist?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Does it have the expected resolution?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Is it base64? URL?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">...
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Now that our API is ready, we need to host it. Maybe with a cloud provider. Several things to worry about here:
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Package API and model in a container
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Where do we deploy it?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">How do we deploy it?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">How do we take advantage of acceleration?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Also:
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">How long do we have to return an answer?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">How many requests per second can we handle?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Do we need automatic scaling?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">What are the criteria to scale in and out?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">How can we tell when a model is down?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">How do we log what happens?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Let& #39;s say we made it.
At this point, we have a frozen, stuck-in-time version of our model deployed.
But we aren& #39;t done yet. Far from it!
By now, there& #39;s probably a newer version of the model ready to go.
How do we deploy that version? Do we need to start again?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
At this point, we have a frozen, stuck-in-time version of our model deployed.
But we aren& #39;t done yet. Far from it!
By now, there& #39;s probably a newer version of the model ready to go.
How do we deploy that version? Do we need to start again?
And of course, it would be ideal if you don& #39;t just snap the new version of the model in and pray that quality doesn& #39;t go down, right?
You want old and new side by side. Then migrate traffic over gradually.
This requires more work.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
You want old and new side by side. Then migrate traffic over gradually.
This requires more work.
Creating the pipeline that handles taking new models and hosting them in production takes a lot of planning and effort.
And you are probably thinking, "That& #39;s MLOps!"
Yes, it is! But giving it a name doesn& #39;t make it less complicated.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
And you are probably thinking, "That& #39;s MLOps!"
Yes, it is! But giving it a name doesn& #39;t make it less complicated.
And there& #39;s more.
As we collect more and more data, we need to train new versions of our model.
We can& #39;t expect our people to do this manually. We need to automate the training pipeline.
A whole lot more work!
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
As we collect more and more data, we need to train new versions of our model.
We can& #39;t expect our people to do this manually. We need to automate the training pipeline.
A whole lot more work!
Some questions:
1. Where& #39;s the data coming from?
2. How should it be split?
3. How much data should be used to retrain?
4. How will the training scripts run?
5. What metrics do we need?
6. How to evaluate the quality of the model?
These aren& #39;t simply "Yes" or "No" answers.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
1. Where& #39;s the data coming from?
2. How should it be split?
3. How much data should be used to retrain?
4. How will the training scripts run?
5. What metrics do we need?
6. How to evaluate the quality of the model?
These aren& #39;t simply "Yes" or "No" answers.
At this point, are we done yet?
Well, not quitehttps://abs.twimg.com/emoji/v2/... draggable="false" alt="😞" title="Enttäuschtes Gesicht" aria-label="Emoji: Enttäuschtes Gesicht">
We need to worry about monitoring our model. How is it performing?
That pesky "concept drift" ensures that the quality of our results will rapidly decay. We need to be on top of it!
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Well, not quite
We need to worry about monitoring our model. How is it performing?
That pesky "concept drift" ensures that the quality of our results will rapidly decay. We need to be on top of it!
And there& #39;s even more.
Here are some must-haves for well-rounded, safe production systems that I haven& #39;t covered yet:
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Ethics
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Data capturing and storage
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Data quality
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Integrating human feedback
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
Here are some must-haves for well-rounded, safe production systems that I haven& #39;t covered yet:
Here is the bottom line:
Creating a model with predictive capacity is just a small part of a much bigger equation.
There aren& #39;t a lot of companies that understand the entire picture. This opens up a lot of opportunities.
Opportunities for you and me.
Creating a model with predictive capacity is just a small part of a much bigger equation.
There aren& #39;t a lot of companies that understand the entire picture. This opens up a lot of opportunities.
Opportunities for you and me.
