Azure Monitor Ingestion Java

1---
2name: azure-monitor-ingestion-java
3description: |
4  Azure Monitor Ingestion SDK for Java. Send custom logs to Azure Monitor via Data Collection Rules (DCR) and Data Collection Endpoints (DCE).
5  Triggers: "LogsIngestionClient java", "azure monitor ingestion java", "custom logs java", "DCR java", "data collection rule java".
6package: com.azure:azure-monitor-ingestion
7---
8 
9# Azure Monitor Ingestion SDK for Java
10 
11Client library for sending custom logs to Azure Monitor using the Logs Ingestion API via Data Collection Rules.
12 
13## Installation
14 
15```xml
16<dependency>
17    <groupId>com.azure</groupId>
18    <artifactId>azure-monitor-ingestion</artifactId>
19    <version>1.2.11</version>
20</dependency>
21```
22 
23Or use Azure SDK BOM:
24 
25```xml
26<dependencyManagement>
27    <dependencies>
28        <dependency>
29            <groupId>com.azure</groupId>
30            <artifactId>azure-sdk-bom</artifactId>
31            <version>{bom_version}</version>
32            <type>pom</type>
33            <scope>import</scope>
34        </dependency>
35    </dependencies>
36</dependencyManagement>
37 
38<dependencies>
39    <dependency>
40        <groupId>com.azure</groupId>
41        <artifactId>azure-monitor-ingestion</artifactId>
42    </dependency>
43</dependencies>
44```
45 
46## Prerequisites
47 
48- Data Collection Endpoint (DCE)
49- Data Collection Rule (DCR)
50- Log Analytics workspace
51- Target table (custom or built-in: CommonSecurityLog, SecurityEvents, Syslog, WindowsEvents)
52 
53## Environment Variables
54 
55```bash
56DATA_COLLECTION_ENDPOINT=https://<dce-name>.<region>.ingest.monitor.azure.com
57DATA_COLLECTION_RULE_ID=dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
58STREAM_NAME=Custom-MyTable_CL
59```
60 
61## Client Creation
62 
63### Synchronous Client
64 
65```java
66import com.azure.identity.DefaultAzureCredential;
67import com.azure.identity.DefaultAzureCredentialBuilder;
68import com.azure.monitor.ingestion.LogsIngestionClient;
69import com.azure.monitor.ingestion.LogsIngestionClientBuilder;
70 
71DefaultAzureCredential credential = new DefaultAzureCredentialBuilder().build();
72 
73LogsIngestionClient client = new LogsIngestionClientBuilder()
74    .endpoint("<data-collection-endpoint>")
75    .credential(credential)
76    .buildClient();
77```
78 
79### Asynchronous Client
80 
81```java
82import com.azure.monitor.ingestion.LogsIngestionAsyncClient;
83 
84LogsIngestionAsyncClient asyncClient = new LogsIngestionClientBuilder()
85    .endpoint("<data-collection-endpoint>")
86    .credential(new DefaultAzureCredentialBuilder().build())
87    .buildAsyncClient();
88```
89 
90## Key Concepts
91 
92| Concept | Description |
93|---------|-------------|
94| Data Collection Endpoint (DCE) | Ingestion endpoint URL for your region |
95| Data Collection Rule (DCR) | Defines data transformation and routing to tables |
96| Stream Name | Target stream in the DCR (e.g., `Custom-MyTable_CL`) |
97| Log Analytics Workspace | Destination for ingested logs |
98 
99## Core Operations
100 
101### Upload Custom Logs
102 
103```java
104import java.util.List;
105import java.util.ArrayList;
106 
107List<Object> logs = new ArrayList<>();
108logs.add(new MyLogEntry("2024-01-15T10:30:00Z", "INFO", "Application started"));
109logs.add(new MyLogEntry("2024-01-15T10:30:05Z", "DEBUG", "Processing request"));
110 
111client.upload("<data-collection-rule-id>", "<stream-name>", logs);
112System.out.println("Logs uploaded successfully");
113```
114 
115### Upload with Concurrency
116 
117For large log collections, enable concurrent uploads:
118 
119```java
120import com.azure.monitor.ingestion.models.LogsUploadOptions;
121import com.azure.core.util.Context;
122 
123List<Object> logs = getLargeLogs(); // Large collection
124 
125LogsUploadOptions options = new LogsUploadOptions()
126    .setMaxConcurrency(3);
127 
128client.upload("<data-collection-rule-id>", "<stream-name>", logs, options, Context.NONE);
129```
130 
131### Upload with Error Handling
132 
133Handle partial upload failures gracefully:
134 
135```java
136LogsUploadOptions options = new LogsUploadOptions()
137    .setLogsUploadErrorConsumer(uploadError -> {
138        System.err.println("Upload error: " + uploadError.getResponseException().getMessage());
139        System.err.println("Failed logs count: " + uploadError.getFailedLogs().size());
140        
141        // Option 1: Log and continue
142        // Option 2: Throw to abort remaining uploads
143        // throw uploadError.getResponseException();
144    });
145 
146client.upload("<data-collection-rule-id>", "<stream-name>", logs, options, Context.NONE);
147```
148 
149### Async Upload with Reactor
150 
151```java
152import reactor.core.publisher.Mono;
153 
154List<Object> logs = getLogs();
155 
156asyncClient.upload("<data-collection-rule-id>", "<stream-name>", logs)
157    .doOnSuccess(v -> System.out.println("Upload completed"))
158    .doOnError(e -> System.err.println("Upload failed: " + e.getMessage()))
159    .subscribe();
160```
161 
162## Log Entry Model Example
163 
164```java
165public class MyLogEntry {
166    private String timeGenerated;
167    private String level;
168    private String message;
169    
170    public MyLogEntry(String timeGenerated, String level, String message) {
171        this.timeGenerated = timeGenerated;
172        this.level = level;
173        this.message = message;
174    }
175    
176    // Getters required for JSON serialization
177    public String getTimeGenerated() { return timeGenerated; }
178    public String getLevel() { return level; }
179    public String getMessage() { return message; }
180}
181```
182 
183## Error Handling
184 
185```java
186import com.azure.core.exception.HttpResponseException;
187 
188try {
189    client.upload(ruleId, streamName, logs);
190} catch (HttpResponseException e) {
191    System.err.println("HTTP Status: " + e.getResponse().getStatusCode());
192    System.err.println("Error: " + e.getMessage());
193    
194    if (e.getResponse().getStatusCode() == 403) {
195        System.err.println("Check DCR permissions and managed identity");
196    } else if (e.getResponse().getStatusCode() == 404) {
197        System.err.println("Verify DCE endpoint and DCR ID");
198    }
199}
200```
201 
202## Best Practices
203 
2041. **Batch logs** — Upload in batches rather than one at a time
2052. **Use concurrency** — Set `maxConcurrency` for large uploads
2063. **Handle partial failures** — Use error consumer to log failed entries
2074. **Match DCR schema** — Log entry fields must match DCR transformation expectations
2085. **Include TimeGenerated** — Most tables require a timestamp field
2096. **Reuse client** — Create once, reuse throughout application
2107. **Use async for high throughput** — `LogsIngestionAsyncClient` for reactive patterns
211 
212## Querying Uploaded Logs
213 
214Use [azure-monitor-query](../query/SKILL.md) to query ingested logs:
215 
216```java
217// See azure-monitor-query skill for LogsQueryClient usage
218String query = "MyTable_CL | where TimeGenerated > ago(1h) | limit 10";
219```
220 
221## Reference Links
222 
223| Resource | URL |
224|----------|-----|
225| Maven Package | https://central.sonatype.com/artifact/com.azure/azure-monitor-ingestion |
226| GitHub | https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/monitor/azure-monitor-ingestion |
227| Product Docs | https://learn.microsoft.com/azure/azure-monitor/logs/logs-ingestion-api-overview |
228| DCE Overview | https://learn.microsoft.com/azure/azure-monitor/essentials/data-collection-endpoint-overview |
229| DCR Overview | https://learn.microsoft.com/azure/azure-monitor/essentials/data-collection-rule-overview |
230| Troubleshooting | https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/monitor/azure-monitor-ingestion/TROUBLESHOOTING.md |
231