Azure Technical Architect and ML Engineer Manager - Remote - $160k
As a Lead ML Engineer and Technical Architect, you will work closely with our data scientists and data engineers design, build and deploy data engineering and machine learning solutions for business usage in a production environment, specifically on Azure. The manager will require deep hands on experience in all phases of a data engineering and machine learning life-cycle to design, harden, build, deploy, scale, tune, monitor these data engineering pipelines and machine learning models for operationalization of solutions using them. You are expected to also mentor and provide guidance to junior engineers and scientists and assist them with design decisions. It is an exciting opportunity to apply your skills to deliver deep data engineering and machine learning enabled business applications to industry problems at scale.
Responsibilities will include:
- Leading large-scale development and operational deployment of Data and ML applications on Azure for business usage
- Leading critical decision making for architecting and designing these applications around technologies, tools, algorithms, libraries, frameworks, deployment infrastructures on Azure Cloud
- Design and package deployments of data engineering pipelines and machine learning models on Azure, for production environments working very closely with data scientists and data engineers.
- Using experimentation code developed by Data Scientists as a basis, design and code production ready high-quality scalable application code for operational deployment.
- Defining the pre-processing or feature engineering to be done on a given dataset and defining data augmentation pipelines on Azure
- Define and incorporate software engineering practices into data engineering and data science model implementation code and deployments on Azure
- Optimize data engineering pipeline performance and model performance and calibrate on appropriate combination of Azure infrastructure and Azure ML tools and services
- Design and Implement APIs serving data requests as well as model outcomes, integrating with business applications, incorporate business rules and obtaining feedback
- Design and implement optimal data and model stores and collection of data and model metadata, using Azure native services, during training and operations
- Design and Implement data engineering pipeline monitoring and model monitoring and calibration solutions on Azure, for performance tracking of production deployed solution
- Design and Implement containerized deployments of ML models and data products to Azure edge infrastructures
- Design and Implement A/B testing approaches and standards for model evaluation
- Bachelor's degree in Computer Science, Engineering, Statistics, Technical Science or 2+ years of IT/Software Development/Programming experience
- Minimum 3+ years' experience of building and deploying production applications for data products and embedded Deep Learning and Machine Learning models
- Minimum 2+ years' experience in designing and deploying production data engineering pipelines and ML models (standalone and distributed) on Azure infrastructure and Azure native services .
- Minimum 2+ years' experience setting and using model parameters and hyperparameters i.e. containerize and externalize to tune and scale the model for large datasets. Must have experience of deploying containerized models and ML pipelines using Docker, Kubernetes or equivalent technologies on Azure
- Minimum 2+ years' experience in engineering models using frameworks such as TensorFlow, Kera's, SciKit, PySpark,OpenCV etc.
- Minimum 3+ years' experience of using Jupyterhub, Anaconda , Spyder, Azure Databricks, Sagemaker,Flask for model engineering, deployments and monitoring.
- Minimum 2+ years' experience using tools like Databricks MLFlow for designing and managing end-to-end machine learning lifecycle for tracking experiments, packaging ML code and deploying models from various ML libraries to model serving and inference platforms
- Minimum 3+ years' experience of building, containerizing and deploying end to end automated data engineering and ML pipelines using technologies like Spark and Azure Native services in a large-scale production environment
- Minimum 3+ years' experience with performance engineering of these models with very large-scale datasets on a large distributed infrastructure using technologies like Azure Synapse analytics, Azure Databricks, Azure API management .
- Experience with Azure ML services as well as third party libraries that support learning models and algorithms.
- Minimum 5+ years of strong programming skills in at least 2 languages from Python, Scala (and Spark), R, Java, C/C++ on Azure
- Minimum 5+ years of deep understanding of software engineering and software architecture principles for building and deploying large-scale business critical applications.
- Minimum 2+ years of understanding and experience in deploying containerized applications using Docker, Kubernetes or equivalent technologies on Azure
- Understanding of multiple Emerging Data and NoSQL Technologies
- Understanding of Data collection and analysis,
- Understanding of all phases of a complete Data Engineering and Dara Science Life-cycle
- Data and Analytics Pipeline Design & Automation
- Knowledge of multiple machine learning and deep learning techniques, technologies, tools
- Understanding of Statistical modeling,
- Strong Experience delivering and deploying medium to large scale solutions
- Knowledge and Understanding of Azure DevOps and MLOps
- Design Thinking
- Prior experience of consulting
- Background in data structures, and object-oriented programming
If this role is of interest, please contact Shannon today at email@example.com