Read on Twitter
Wanna know what a day in the life of a Machine Learning Engineer looks like?
I’m not gonna make this boring and talk about biking, coffee, or hipster stuff. Instead, I’ll list some of the things that you may find yourself doing.
This is a thread
https://abs.twimg.com/emoji/v2/... draggable="false" alt="🧵" title="Thread" aria-label="Emoji: Thread"> full of specifics.
I’m not gonna make this boring and talk about biking, coffee, or hipster stuff. Instead, I’ll list some of the things that you may find yourself doing.
This is a thread
Different companies may decide to distribute these tasks across different roles. Please, don’t @ me saying that an MLE isn’t supposed to do this or that.
Let’s start!
1. Discussed with a client the characteristics of their existing data and designed a plan to augment it using synthetic data.
2. Implemented a Python script to generate synthetic data based on a template image and a set of rules.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
1. Discussed with a client the characteristics of their existing data and designed a plan to augment it using synthetic data.
2. Implemented a Python script to generate synthetic data based on a template image and a set of rules.
3. Set up a pipeline using AWS Step Functions, Lambdas, and SQS to process a large dataset of images sitting in an S3 bucket.
4. Implemented a synchronization of an S3 bucket with a local file system and a MySQL table. Developed in Python, using AWS SQS, and Lambdas.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
4. Implemented a synchronization of an S3 bucket with a local file system and a MySQL table. Developed in Python, using AWS SQS, and Lambdas.
5. Designed and implemented a Docker image to deploy TensorFlow Object Detection models to SageMaker endpoints.
6. Designed and implemented a set of scripts to train and deploy different models to a SageMaker multi-model endpoint.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
6. Designed and implemented a set of scripts to train and deploy different models to a SageMaker multi-model endpoint.
7. Designed a Computer Vision model that uses a custom Keras generator, and combines pixel data with additional features to improve its performance.
8. Implemented an app that connects to @BostonDynamics’s Spot cameras, and makes the robot react to visual clues.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
8. Implemented an app that connects to @BostonDynamics’s Spot cameras, and makes the robot react to visual clues.
9. Worked on a front-end, Angular application to implement socketio notifications between the app and the server.
10. Implemented a Flask API to allow for an application running on Fargate to create SageMaker Labeling Jobs.
(The list keeps going but you get the idea)
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
10. Implemented a Flask API to allow for an application running on Fargate to create SageMaker Labeling Jobs.
(The list keeps going but you get the idea)
As you can see, there are a lot of different things going on this list.
I mentioned it before: Larger companies usually split this work across multi-disciplinary teams. Smaller companies do what they have to do.
I like to be stretched like this. But that’s just me.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
I mentioned it before: Larger companies usually split this work across multi-disciplinary teams. Smaller companies do what they have to do.
I like to be stretched like this. But that’s just me.
Let’s go over some of the main skills and tools that help me do my job:
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Python (and a little bit of JavaScript)
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Docker, Fargate, Kubernetes
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Flask + gunicorn + nginx
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">OpenCV
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">TensorFlow + Keras, Scikit-Learn
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">Jupyter Notebooks, Google Colab
https://abs.twimg.com/emoji/v2/... draggable="false" alt="▫️" title="Weißes kleines Quadrat" aria-label="Emoji: Weißes kleines Quadrat">MySQL, DynamoDB
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
It looks like a lot, but I’m mediocre at best in most of these. I’ve learned by doing, and enough to provide value.
You do not need to get a Ph.D. in Python to rock somebody’s world. You do not need to be an expert in anything. Aim for “good enough” and go from there.
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
You do not need to get a Ph.D. in Python to rock somebody’s world. You do not need to be an expert in anything. Aim for “good enough” and go from there.
If you are a Machine Learning Engineer, leave me a comment and tell me what do you differently.
What interesting projects are you working on? I’d love to geek out with you, so show me something!
What interesting projects are you working on? I’d love to geek out with you, so show me something!
