As a researcher, I used to think that beating state of the art is the most important thing in machine learning.

Now, with a bit more experience, I know that I was wrong.

This is a thread about a few other critical points that are often not considered quite enough!

https://abs.twimg.com/emoji/v2/... draggable="false" alt="1️⃣" title="Tastenkappe Ziffer 1" aria-label="Emoji: Tastenkappe Ziffer 1"> Data engineering.

As the saying goes: garbage in, garbage out. Data is more important than the model itself!

https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Does the training data reflect the problem well?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Is it balanced?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Does it contain bias?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Do you have a good test set?

https://abs.twimg.com/emoji/v2/... draggable="false" alt="2️⃣" title="Tastenkappe Ziffer 2" aria-label="Emoji: Tastenkappe Ziffer 2"> User-friendly interface.

Models are trained to be used by other developers and services.

https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Does it have a well-documented API?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Is it straightforward to deploy and use?

https://abs.twimg.com/emoji/v2/... draggable="false" alt="3️⃣" title="Tastenkappe Ziffer 3" aria-label="Emoji: Tastenkappe Ziffer 3"> Deployment to production.

In software engineering, the faster you can push changes to production, the faster you can move. This is true machine learning for as well.

https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Do you have a robust and automated deployment pipeline?
https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Can you handle if a new model performs worse?

https://abs.twimg.com/emoji/v2/... draggable="false" alt="4️⃣" title="Tastenkappe Ziffer 4" aria-label="Emoji: Tastenkappe Ziffer 4"> Testing and monitoring.

Testing is not a one and done thing. Performance should be monitored constantly. Occasionally, the data coming from production can slowly drift from the training data.

https://abs.twimg.com/emoji/v2/... draggable="false" alt="➡️" title="Pfeil nach rechts" aria-label="Emoji: Pfeil nach rechts"> Is your infrastructure ready to spot these?

This is just the tip of the iceberg. There is so much beyond training a model!

What are the key questions you ask when deploying models to production?