LearnAI - Machine

Learning on Azure
Seth Mottaghinejad
Wolfgang Pauli, PhD

[email protected]
Overview
● Machine Learning on Azure
● Custom AI
● Compute Targets (DSVMs and Managed Compute)
● DevOps for Machine Learning
● Azure Machine Learning Pipelines
● Flexible and Support for Open Source Frameworks
● Deployment
● Tool Agnostic Python SDK
Building blocks for a Data Science Project

Build and train Hyperparameter Experimentation and DevOps for data

models Deployment science
tuning pipelines

Classical ML

Data
sources

Deep learning
Machine Learning on Azure
…
Domain Specific Pretrained Models
To reduce time to market Vision Speech Language Search

Familiar Data Science Tools

To simplify model development PyCharm Jupyter Visual Studio Code Command line

Popular Frameworks
To build machine learning and deep learning solutions PyTorch TensorFlow Scikit-Learn ONNX

Productive Services
To empower data science and development teams Azure
Databricks
Azure Machine
Learning
Machine
Learning VMs

Powerful Hardware
To accelerate deep learning CPU GPU FPGA

From the Intelligent Cloud to the Intelligent Edge

Azure Machine Learning Service

Set of Azure Cloud Python

Services SDK

That enables you to:

✓ Prepare Data ✓ Manage Models
✓ Build Models ✓ Track Experiments
✓ Train Models ✓ Deploy Models
Custom AI
Productive Services
Empower data science and development teams

Integrated data science & data engineering teams Individual data scientists
Desktop solutions not adequate Desktop solutions adequate
Need a unified big data & machine learning solution Need cloud for sporadic compute needs

+
Azure Databricks Azure Machine Learning Machine Learning VMs
service
Data Science Virtual Machine
Data Science
Virtual Machines
(DSVM)
Pre-Configured environments in the
cloud for
Data Science & AI Modeling,
Development & Deployment.

Samples to get started

Managed Compute
Distributed training on managed compute
Dependencies and Containers

Handling failures Schedule jobs

Secure Access Distribute data

Gather results Scale resources

Provision VM clusters
Training Infrastructure

Dependencies and Containers Distribute data

Leverage system-managed AML Manage and share resources across
compute or bring your own a workspace
compute

Schedule jobs Scale resources

Train at cloud scale using a Autoscale resources to only pay while
framework of choice running a job

Provision VM clusters
Use the latest NDv2 series VMs
with the NVIDIA V100 GPUs
Powerful Infrastructure
Accelerate deep learning

CPUs GPUs FPGAs

General purpose machine Deep learning Specialized hardware
learning accelerated deep learning
D, F, L, M, H Series N Series Project Brainwave

Optimized for flexibility Optimized for performance

FPGA NEW UPDATES:

Support for image classification and recognition scenarios
ResNet 50, ResNet 152, VGG-16, SSD-VGG, DenseNet-121
DevOps for Machine Learning
DevOps loop for data science
Prepare Experiment Deploy

Prepare Data Build model Train & Register and Build Deploy Service
(your favorite IDE) Test Model Manage Model Image Monitor Model
DevOps loop for data science
Prepare Train &
Test Model

Register and
… Manage Model
010101001010010111011010010010010110101
001000100011110100101010100101001010100
101001011101101001001001011010100100010
001111010010101010010100010100101110110
100100100101101010010001000111101001010
10100101001010100101001011101101001

Build model
(your favorite IDE)

Prepare Data

Deploy Service
Monitor Model Build
Image
Model Management in detail

Create/Retrain Model Register Model Monitor

Enable DevOps with full CI/CD Track model versions with a Oversea deployments through
integration with VSTS central model registry Azure AppInsights
Experimentation

Leverage service-side capture of run metrics, Manage training jobs locally, scaled-up or
output logs and models scaled-out

80%
75%

90% 95%

85%

Use leaderboards, side by side run comparison Conduct a hyperparameter search on traditional
and model selection ML or DNN
Azure Machine Learning Pipelines
Azure Machine Learning Pipelines
Prepare data Build & train models Deploy & predict

Data Preparation Model building & training Model deployment

Data ingestion

Normalization Hyper-parameter tuning Deployment

Transformation Automatic model selection Batch scoring

Validation Model testing

Data storage Featurization Model validation

locations
Azure Machine Learning Pipelines
Prepare data Build & train models Deploy & predict

Data Preparation Model building & training Model deployment

Data ingestion

Normalization
Normalization Hyper-parameter
Hyper-parameter tuning
tuning Deployment

Transformation
Transformation Automatic
Automatic model
model selection
selection Batch scoring

Validation
Validation Model
Model testing
testing Testing error

Data storage Featurization

Featurization Model validation
locations
Azure Machine Learning Pipelines
Prepare data Build & train models Deploy & predict

Data Preparation Model building & training Model deployment

Data ingestion

Normalization
Normalization Hyper-parameter
Hyper-parameter tuning
tuning Deployment
Deployment

Transformation
Transformation Automatic
Automatic model
model selection
selection Batch
Batch scoring
scoring

Validation
Validation Model
Model testing
testing Testing error

Data storage Featurization

Featurization Model
Model validation
validation
locations Error resolved
Azure Machine Learning Pipelines with new
data
Prepare data Build & train models Deploy & predict

Data Preparation Model building & training Model deployment

Data ingestion

Normalization
Normalization Hyper-parameter
Hyper-parameter tuning
tuning Deployment
Deployment
New
data
Transformation
Transformation Automatic
Automatic model
model selection
selection Batch
Batch scoring
scoring

Validation
Validation Model
Model testing
testing

Data storage Featurization

Featurization Model
Model validation
validation
locations
Advantages of Azure ML Pipelines

Unattended runs Tracking and versioning

Schedule a few steps to run in parallel or in Name and version your data sources, inputs
sequence to focus on other tasks while your and outputs with the pipelines SDK
pipeline runs

Reusability Mixed and diverse compute

Create templates of pipelines for specific Use multiple pipelines that are reliably
scenarios such as retraining and batch scoring coordinated across heterogeneous and
scalable computes and storages
Support for
Open source frameworks
Popular Frameworks

Use your favorite machine learning frameworks without getting locked into one framework

TensorFlow PyTorch Scikit-Learn ONNX

Community project created by Facebook and Microsoft

Use the best tool for the job. Train in one framework and
transfer to another for inference
MXNet Chainer Keras
Create Deploy
Frameworks Azure
Azure Machine Learning services
Native
support Ubuntu VM
Native
support Windows Server 2019 VM

ML.NE Windows Devices

T
Converters
ONNX Model

Services
Native Converters Other Devices
support
Azure Custom (iOS, etc)
Vision Service
Deployment
Flexible Deployment
Deploy and manage models on intelligent cloud and edge

Train & deploy Train & deploy

Model optimization for cloud & edge

Manage models in production
Capture model telemetry
Retrain models

Deploy
Deploy Azure ML models at scale
Azure Machine Learning Service

Cognitive
Services
Image Registry Image Registry
Model Registry Your IDE
Cloud Service

Azure Machine
Learning Heavy Edge
Experimentation

Create & Deployment & Model

Register Model Scoring File Register Image Monitoring
Light Edge

External
Model
Deployments to Compute Targets
Tool Agnostic Python SDK
Tool Agnostic Python SDK

PyCharm Jupyter Visual Studio Code

Use your favorite IDEs, editors, notebooks, and Flexibility of your local environment or
frameworks curated cloud environment

Integrate with other services like Get started quickly without any complex
Azure Databricks pre-requisites
Summary
Azure Machine Learning service
Bring AI to everyone with an end-to-end, scalable, trusted platform

Built with your needs in mind

Boost your data science productivity

Automated machine learning

Managed compute

Increase your rate of experimentation

Simple deployment

DevOps for machine learning

Support for open source frameworks

Deploy and manage your models everywhere

Tool agnostic Python SDK

Seamlessly integrated with the Azure Portfolio

Resources beyond this AI Airlift

Azure Machine Learning with Azure Databricks

https://aka.ms/aml-notebook-databricks-e2e

Azure Notebooks
https://notebooks.azure.com/azureml/projects/azureml-getting-started

Azure ML Docs  
https://docs.microsoft.com/en-us/azure/machine-learning/service/
Questions
Resources for this Airlift

Azure Subscriptions 
https://aka.ms/learnAIsubscriptions

Azure Databricks Notebooks

https://aka.ms/learnAInotebooks.dbc

Git Repository for LearnAI CustomAI Partner Airlift

https://github.com/azure/learnai_azure_ml