Azure AI Contentsafety Ts

1---
2name: azure-ai-contentsafety-ts
3description: Analyze text and images for harmful content using Azure AI Content Safety (@azure-rest/ai-content-safety). Use when moderating user-generated content, detecting hate speech, violence, sexual content, or self-harm, or managing custom blocklists.
4package: "@azure-rest/ai-content-safety"
5---
6 
7# Azure AI Content Safety REST SDK for TypeScript
8 
9Analyze text and images for harmful content with customizable blocklists.
10 
11## Installation
12 
13```bash
14npm install @azure-rest/ai-content-safety @azure/identity @azure/core-auth
15```
16 
17## Environment Variables
18 
19```bash
20CONTENT_SAFETY_ENDPOINT=https://<resource>.cognitiveservices.azure.com
21CONTENT_SAFETY_KEY=<api-key>
22```
23 
24## Authentication
25 
26**Important**: This is a REST client. `ContentSafetyClient` is a **function**, not a class.
27 
28### API Key
29 
30```typescript
31import ContentSafetyClient from "@azure-rest/ai-content-safety";
32import { AzureKeyCredential } from "@azure/core-auth";
33 
34const client = ContentSafetyClient(
35  process.env.CONTENT_SAFETY_ENDPOINT!,
36  new AzureKeyCredential(process.env.CONTENT_SAFETY_KEY!)
37);
38```
39 
40### DefaultAzureCredential
41 
42```typescript
43import ContentSafetyClient from "@azure-rest/ai-content-safety";
44import { DefaultAzureCredential } from "@azure/identity";
45 
46const client = ContentSafetyClient(
47  process.env.CONTENT_SAFETY_ENDPOINT!,
48  new DefaultAzureCredential()
49);
50```
51 
52## Analyze Text
53 
54```typescript
55import ContentSafetyClient, { isUnexpected } from "@azure-rest/ai-content-safety";
56 
57const result = await client.path("/text:analyze").post({
58  body: {
59    text: "Text content to analyze",
60    categories: ["Hate", "Sexual", "Violence", "SelfHarm"],
61    outputType: "FourSeverityLevels"  // or "EightSeverityLevels"
62  }
63});
64 
65if (isUnexpected(result)) {
66  throw result.body;
67}
68 
69for (const analysis of result.body.categoriesAnalysis) {
70  console.log(`${analysis.category}: severity ${analysis.severity}`);
71}
72```
73 
74## Analyze Image
75 
76### Base64 Content
77 
78```typescript
79import { readFileSync } from "node:fs";
80 
81const imageBuffer = readFileSync("./image.png");
82const base64Image = imageBuffer.toString("base64");
83 
84const result = await client.path("/image:analyze").post({
85  body: {
86    image: { content: base64Image }
87  }
88});
89 
90if (isUnexpected(result)) {
91  throw result.body;
92}
93 
94for (const analysis of result.body.categoriesAnalysis) {
95  console.log(`${analysis.category}: severity ${analysis.severity}`);
96}
97```
98 
99### Blob URL
100 
101```typescript
102const result = await client.path("/image:analyze").post({
103  body: {
104    image: { blobUrl: "https://storage.blob.core.windows.net/container/image.png" }
105  }
106});
107```
108 
109## Blocklist Management
110 
111### Create Blocklist
112 
113```typescript
114const result = await client
115  .path("/text/blocklists/{blocklistName}", "my-blocklist")
116  .patch({
117    contentType: "application/merge-patch+json",
118    body: {
119      description: "Custom blocklist for prohibited terms"
120    }
121  });
122 
123if (isUnexpected(result)) {
124  throw result.body;
125}
126 
127console.log(`Created: ${result.body.blocklistName}`);
128```
129 
130### Add Items to Blocklist
131 
132```typescript
133const result = await client
134  .path("/text/blocklists/{blocklistName}:addOrUpdateBlocklistItems", "my-blocklist")
135  .post({
136    body: {
137      blocklistItems: [
138        { text: "prohibited-term-1", description: "First blocked term" },
139        { text: "prohibited-term-2", description: "Second blocked term" }
140      ]
141    }
142  });
143 
144if (isUnexpected(result)) {
145  throw result.body;
146}
147 
148for (const item of result.body.blocklistItems ?? []) {
149  console.log(`Added: ${item.blocklistItemId}`);
150}
151```
152 
153### Analyze with Blocklist
154 
155```typescript
156const result = await client.path("/text:analyze").post({
157  body: {
158    text: "Text that might contain blocked terms",
159    blocklistNames: ["my-blocklist"],
160    haltOnBlocklistHit: false
161  }
162});
163 
164if (isUnexpected(result)) {
165  throw result.body;
166}
167 
168// Check blocklist matches
169if (result.body.blocklistsMatch) {
170  for (const match of result.body.blocklistsMatch) {
171    console.log(`Blocked: "${match.blocklistItemText}" from ${match.blocklistName}`);
172  }
173}
174```
175 
176### List Blocklists
177 
178```typescript
179const result = await client.path("/text/blocklists").get();
180 
181if (isUnexpected(result)) {
182  throw result.body;
183}
184 
185for (const blocklist of result.body.value ?? []) {
186  console.log(`${blocklist.blocklistName}: ${blocklist.description}`);
187}
188```
189 
190### Delete Blocklist
191 
192```typescript
193await client.path("/text/blocklists/{blocklistName}", "my-blocklist").delete();
194```
195 
196## Harm Categories
197 
198| Category | API Term | Description |
199|----------|----------|-------------|
200| Hate and Fairness | `Hate` | Discriminatory language targeting identity groups |
201| Sexual | `Sexual` | Sexual content, nudity, pornography |
202| Violence | `Violence` | Physical harm, weapons, terrorism |
203| Self-Harm | `SelfHarm` | Self-injury, suicide, eating disorders |
204 
205## Severity Levels
206 
207| Level | Risk | Recommended Action |
208|-------|------|-------------------|
209| 0 | Safe | Allow |
210| 2 | Low | Review or allow with warning |
211| 4 | Medium | Block or require human review |
212| 6 | High | Block immediately |
213 
214**Output Types**:
215- `FourSeverityLevels` (default): Returns 0, 2, 4, 6
216- `EightSeverityLevels`: Returns 0-7
217 
218## Content Moderation Helper
219 
220```typescript
221import ContentSafetyClient, { 
222  isUnexpected, 
223  TextCategoriesAnalysisOutput 
224} from "@azure-rest/ai-content-safety";
225 
226interface ModerationResult {
227  isAllowed: boolean;
228  flaggedCategories: string[];
229  maxSeverity: number;
230  blocklistMatches: string[];
231}
232 
233async function moderateContent(
234  client: ReturnType<typeof ContentSafetyClient>,
235  text: string,
236  maxAllowedSeverity = 2,
237  blocklistNames: string[] = []
238): Promise<ModerationResult> {
239  const result = await client.path("/text:analyze").post({
240    body: { text, blocklistNames, haltOnBlocklistHit: false }
241  });
242 
243  if (isUnexpected(result)) {
244    throw result.body;
245  }
246 
247  const flaggedCategories = result.body.categoriesAnalysis
248    .filter(c => (c.severity ?? 0) > maxAllowedSeverity)
249    .map(c => c.category!);
250 
251  const maxSeverity = Math.max(
252    ...result.body.categoriesAnalysis.map(c => c.severity ?? 0)
253  );
254 
255  const blocklistMatches = (result.body.blocklistsMatch ?? [])
256    .map(m => m.blocklistItemText!);
257 
258  return {
259    isAllowed: flaggedCategories.length === 0 && blocklistMatches.length === 0,
260    flaggedCategories,
261    maxSeverity,
262    blocklistMatches
263  };
264}
265```
266 
267## API Endpoints
268 
269| Operation | Method | Path |
270|-----------|--------|------|
271| Analyze Text | POST | `/text:analyze` |
272| Analyze Image | POST | `/image:analyze` |
273| Create/Update Blocklist | PATCH | `/text/blocklists/{blocklistName}` |
274| List Blocklists | GET | `/text/blocklists` |
275| Delete Blocklist | DELETE | `/text/blocklists/{blocklistName}` |
276| Add Blocklist Items | POST | `/text/blocklists/{blocklistName}:addOrUpdateBlocklistItems` |
277| List Blocklist Items | GET | `/text/blocklists/{blocklistName}/blocklistItems` |
278| Remove Blocklist Items | POST | `/text/blocklists/{blocklistName}:removeBlocklistItems` |
279 
280## Key Types
281 
282```typescript
283import ContentSafetyClient, {
284  isUnexpected,
285  AnalyzeTextParameters,
286  AnalyzeImageParameters,
287  TextCategoriesAnalysisOutput,
288  ImageCategoriesAnalysisOutput,
289  TextBlocklist,
290  TextBlocklistItem
291} from "@azure-rest/ai-content-safety";
292```
293 
294## Best Practices
295 
2961. **Always use isUnexpected()** - Type guard for error handling
2972. **Set appropriate thresholds** - Different categories may need different severity thresholds
2983. **Use blocklists for domain-specific terms** - Supplement AI detection with custom rules
2994. **Log moderation decisions** - Keep audit trail for compliance
3005. **Handle edge cases** - Empty text, very long text, unsupported image formats
301