Read on Twitter
As a researcher, I used to think that beating state of the art is the most important thing in machine learning.
Now, with a bit more experience, I know that I was wrong.
This is a thread about a few other critical points that are often not considered quite enough!
Now, with a bit more experience, I know that I was wrong.
This is a thread about a few other critical points that are often not considered quite enough!
Data engineering.As the saying goes: garbage in, garbage out. Data is more important than the model itself!
Does the training data reflect the problem well? Is it balanced? Does it contain bias? Do you have a good test set?
User-friendly interface.Models are trained to be used by other developers and services.
Does it have a well-documented API? Is it straightforward to deploy and use?
Deployment to production.In software engineering, the faster you can push changes to production, the faster you can move. This is true machine learning for as well.
Do you have a robust and automated deployment pipeline? Can you handle if a new model performs worse?
Testing and monitoring.Testing is not a one and done thing. Performance should be monitored constantly. Occasionally, the data coming from production can slowly drift from the training data.
Is your infrastructure ready to spot these?
This is just the tip of the iceberg. There is so much beyond training a model!
What are the key questions you ask when deploying models to production?
What are the key questions you ask when deploying models to production?
