Azure AI ML Py

1---
2name: azure-ai-ml-py
3description: |
4  Azure Machine Learning SDK v2 for Python. Use for ML workspaces, jobs, models, datasets, compute, and pipelines.
5  Triggers: "azure-ai-ml", "MLClient", "workspace", "model registry", "training jobs", "datasets".
6package: azure-ai-ml
7---
8 
9# Azure Machine Learning SDK v2 for Python
10 
11Client library for managing Azure ML resources: workspaces, jobs, models, data, and compute.
12 
13## Installation
14 
15```bash
16pip install azure-ai-ml
17```
18 
19## Environment Variables
20 
21```bash
22AZURE_SUBSCRIPTION_ID=<your-subscription-id>
23AZURE_RESOURCE_GROUP=<your-resource-group>
24AZURE_ML_WORKSPACE_NAME=<your-workspace-name>
25```
26 
27## Authentication
28 
29```python
30from azure.ai.ml import MLClient
31from azure.identity import DefaultAzureCredential
32 
33ml_client = MLClient(
34    credential=DefaultAzureCredential(),
35    subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"],
36    resource_group_name=os.environ["AZURE_RESOURCE_GROUP"],
37    workspace_name=os.environ["AZURE_ML_WORKSPACE_NAME"]
38)
39```
40 
41### From Config File
42 
43```python
44from azure.ai.ml import MLClient
45from azure.identity import DefaultAzureCredential
46 
47# Uses config.json in current directory or parent
48ml_client = MLClient.from_config(
49    credential=DefaultAzureCredential()
50)
51```
52 
53## Workspace Management
54 
55### Create Workspace
56 
57```python
58from azure.ai.ml.entities import Workspace
59 
60ws = Workspace(
61    name="my-workspace",
62    location="eastus",
63    display_name="My Workspace",
64    description="ML workspace for experiments",
65    tags={"purpose": "demo"}
66)
67 
68ml_client.workspaces.begin_create(ws).result()
69```
70 
71### List Workspaces
72 
73```python
74for ws in ml_client.workspaces.list():
75    print(f"{ws.name}: {ws.location}")
76```
77 
78## Data Assets
79 
80### Register Data
81 
82```python
83from azure.ai.ml.entities import Data
84from azure.ai.ml.constants import AssetTypes
85 
86# Register a file
87my_data = Data(
88    name="my-dataset",
89    version="1",
90    path="azureml://datastores/workspaceblobstore/paths/data/train.csv",
91    type=AssetTypes.URI_FILE,
92    description="Training data"
93)
94 
95ml_client.data.create_or_update(my_data)
96```
97 
98### Register Folder
99 
100```python
101my_data = Data(
102    name="my-folder-dataset",
103    version="1",
104    path="azureml://datastores/workspaceblobstore/paths/data/",
105    type=AssetTypes.URI_FOLDER
106)
107 
108ml_client.data.create_or_update(my_data)
109```
110 
111## Model Registry
112 
113### Register Model
114 
115```python
116from azure.ai.ml.entities import Model
117from azure.ai.ml.constants import AssetTypes
118 
119model = Model(
120    name="my-model",
121    version="1",
122    path="./model/",
123    type=AssetTypes.CUSTOM_MODEL,
124    description="My trained model"
125)
126 
127ml_client.models.create_or_update(model)
128```
129 
130### List Models
131 
132```python
133for model in ml_client.models.list(name="my-model"):
134    print(f"{model.name} v{model.version}")
135```
136 
137## Compute
138 
139### Create Compute Cluster
140 
141```python
142from azure.ai.ml.entities import AmlCompute
143 
144cluster = AmlCompute(
145    name="cpu-cluster",
146    type="amlcompute",
147    size="Standard_DS3_v2",
148    min_instances=0,
149    max_instances=4,
150    idle_time_before_scale_down=120
151)
152 
153ml_client.compute.begin_create_or_update(cluster).result()
154```
155 
156### List Compute
157 
158```python
159for compute in ml_client.compute.list():
160    print(f"{compute.name}: {compute.type}")
161```
162 
163## Jobs
164 
165### Command Job
166 
167```python
168from azure.ai.ml import command, Input
169 
170job = command(
171    code="./src",
172    command="python train.py --data ${{inputs.data}} --lr ${{inputs.learning_rate}}",
173    inputs={
174        "data": Input(type="uri_folder", path="azureml:my-dataset:1"),
175        "learning_rate": 0.01
176    },
177    environment="AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest",
178    compute="cpu-cluster",
179    display_name="training-job"
180)
181 
182returned_job = ml_client.jobs.create_or_update(job)
183print(f"Job URL: {returned_job.studio_url}")
184```
185 
186### Monitor Job
187 
188```python
189ml_client.jobs.stream(returned_job.name)
190```
191 
192## Pipelines
193 
194```python
195from azure.ai.ml import dsl, Input, Output
196from azure.ai.ml.entities import Pipeline
197 
198@dsl.pipeline(
199    compute="cpu-cluster",
200    description="Training pipeline"
201)
202def training_pipeline(data_input):
203    prep_step = prep_component(data=data_input)
204    train_step = train_component(
205        data=prep_step.outputs.output_data,
206        learning_rate=0.01
207    )
208    return {"model": train_step.outputs.model}
209 
210pipeline = training_pipeline(
211    data_input=Input(type="uri_folder", path="azureml:my-dataset:1")
212)
213 
214pipeline_job = ml_client.jobs.create_or_update(pipeline)
215```
216 
217## Environments
218 
219### Create Custom Environment
220 
221```python
222from azure.ai.ml.entities import Environment
223 
224env = Environment(
225    name="my-env",
226    version="1",
227    image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
228    conda_file="./environment.yml"
229)
230 
231ml_client.environments.create_or_update(env)
232```
233 
234## Datastores
235 
236### List Datastores
237 
238```python
239for ds in ml_client.datastores.list():
240    print(f"{ds.name}: {ds.type}")
241```
242 
243### Get Default Datastore
244 
245```python
246default_ds = ml_client.datastores.get_default()
247print(f"Default: {default_ds.name}")
248```
249 
250## MLClient Operations
251 
252| Property | Operations |
253|----------|------------|
254| `workspaces` | create, get, list, delete |
255| `jobs` | create_or_update, get, list, stream, cancel |
256| `models` | create_or_update, get, list, archive |
257| `data` | create_or_update, get, list |
258| `compute` | begin_create_or_update, get, list, delete |
259| `environments` | create_or_update, get, list |
260| `datastores` | create_or_update, get, list, get_default |
261| `components` | create_or_update, get, list |
262 
263## Best Practices
264 
2651. **Use versioning** for data, models, and environments
2662. **Configure idle scale-down** to reduce compute costs
2673. **Use environments** for reproducible training
2684. **Stream job logs** to monitor progress
2695. **Register models** after successful training jobs
2706. **Use pipelines** for multi-step workflows
2717. **Tag resources** for organization and cost tracking
272