Skip to content

Commit 9c250fe

Browse files
zhaiyutongzhaiyutong
authored
Add Azure Content Safety java (Azure#36736)
1 parent e7f7ff0 commit 9c250fe

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+6091
-0
lines changed

‎eng/versioning/version_client.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,7 @@ com.azure:azure-communication-email;1.0.5;1.1.0-beta.1
189189
com.azure:azure-developer-loadtesting;1.0.5;1.1.0-beta.1
190190
com.azure:azure-identity-extensions;1.1.7;1.2.0-beta.2
191191
com.azure:azure-analytics-purview-sharing;1.0.0-beta.2;1.0.0-beta.3
192+
com.azure:azure-ai-contentsafety;1.0.0-beta.1;1.0.0-beta.1
192193
com.azure.spring:azure-monitor-spring-native;1.0.0-beta.1;1.0.0-beta.1
193194
com.azure.spring:azure-monitor-spring-native-test;1.0.0-beta.1;1.0.0-beta.1
194195
com.azure.spring:spring-cloud-azure-appconfiguration-config-web;4.11.0;4.12.0-beta.1

‎pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
<module>sdk/consumption</module>
5050
<module>sdk/containerregistry</module>
5151
<module>sdk/containerservicefleet</module>
52+
<module>sdk/contentsafety</module>
5253
<module>sdk/core</module>
5354
<module>sdk/cosmos</module>
5455
<module>sdk/cosmosdbforpostgresql</module>

‎sdk/contentsafety/azure-ai-contentsafety/CHANGELOG.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Release History
2+
3+
## 1.0.0-beta.1 (Unreleased)
4+
5+
- Azure AI ContentSafety client library for Java. This package contains Microsoft Azure ContentSafety client library.
6+
7+
### Features Added
8+
* Text Analysis API: Scans text for sexual content, violence, hate, and self harm with multi-severity levels.
9+
* Image Analysis API: Scans images for sexual content, violence, hate, and self harm with multi-severity levels.
10+
* Text Blocklist Management APIs: The default AI classifiers are sufficient for most content safety needs; however, you might need to screen for terms that are specific to your use case. You can create blocklists of terms to use with the Text API.
11+
12+
### Breaking Changes
13+
14+
### Bugs Fixed
15+
16+
### Other Changes

‎sdk/contentsafety/azure-ai-contentsafety/README.md‎

Lines changed: 288 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,288 @@
1+
# Azure ContentSafety client library for Java
2+
3+
[Azure AI Content Safety][contentsafety_overview] detects harmful user-generated and AI-generated content in applications and services. Content Safety includes several APIs that allow you to detect material that is harmful:
4+
5+
* Text Analysis API: Scans text for sexual content, violence, hate, and self harm with multi-severity levels.
6+
* Image Analysis API: Scans images for sexual content, violence, hate, and self harm with multi-severity levels.
7+
* Text Blocklist Management APIs: The default AI classifiers are sufficient for most content safety needs; however, you might need to screen for terms that are specific to your use case. You can create blocklists of terms to use with the Text API.
8+
9+
## Documentation
10+
11+
Various documentation is available to help you get started
12+
13+
- [API reference documentation][docs]
14+
- [Product documentation][product_documentation]
15+
16+
## Getting started
17+
18+
### Prerequisites
19+
20+
- [Java Development Kit (JDK)][jdk] with version 8 or above
21+
- You need an [Azure subscription][azure_sub] to use this package.
22+
- An existing [Azure AI Content Safety][contentsafety_overview] instance.
23+
24+
### Adding the package to your product
25+
26+
[//]: # ({x-version-update-start;com.azure:azure-ai-contentsafety;current})
27+
```xml
28+
<dependency>
29+
<groupId>com.azure</groupId>
30+
<artifactId>azure-ai-contentsafety</artifactId>
31+
<version>1.0.0-beta.1</version>
32+
</dependency>
33+
```
34+
[//]: # ({x-version-update-end})
35+
36+
### Authenticate the client
37+
38+
#### Get the endpoint
39+
You can find the endpoint for your Azure AI Content Safety service resource using the [Azure Portal][azure_portal] or [Azure CLI][azure_cli_endpoint_lookup]:
40+
41+
```bash
42+
# Get the endpoint for the Azure AI Content Safety service resource
43+
az cognitiveservices account show --name "<resource-name>" --resource-group "<resource-group-name>" --query "properties.endpoint"
44+
```
45+
46+
#### Get the API key
47+
48+
The API key can be found in the [Azure Portal][azure_portal] or by running the following [Azure CLI][azure_cli_key_lookup] command:
49+
50+
```bash
51+
az cognitiveservices account keys list --name "<resource-name>" --resource-group "<resource-group-name>"
52+
```
53+
#### Create a ContentSafetyClient with KeyCredential
54+
```java com.azure.ai.contentsafety.createClient
55+
String endpoint = Configuration.getGlobalConfiguration().get("CONTENT_SAFETY_ENDPOINT");
56+
String key = Configuration.getGlobalConfiguration().get("CONTENT_SAFETY_KEY");
57+
58+
ContentSafetyClient contentSafetyClient = new ContentSafetyClientBuilder()
59+
.credential(new KeyCredential(key))
60+
.endpoint(endpoint).buildClient();
61+
```
62+
63+
## Key concepts
64+
### Harm categories
65+
66+
Content Safety recognizes four distinct categories of objectionable content.
67+
68+
|Category |Description |
69+
|---------|---------|
70+
|Hate |Hate refers to any content that attacks or uses pejorative or discriminatory language in reference to a person or identity group based on certain differentiating attributes of that group. This includes but is not limited to race, ethnicity, nationality, gender identity and expression, sexual orientation, religion, immigration status, ability status, personal appearance, and body size.|
71+
|Sexual |Sexual describes content related to anatomical organs and genitals, romantic relationships, acts portrayed in erotic or affectionate terms, pregnancy, physical sexual acts—including those acts portrayed as an assault or a forced sexual violent act against one’s will—, prostitution, pornography, and abuse.|
72+
|Violence |Violence describes content related to physical actions intended to hurt, injure, damage, or kill someone or something. It also includes weapons, guns and related entities, such as manufacturers, associations, legislation, and similar.|
73+
|Self-harm |Self-harm describes content related to physical actions intended to purposely hurt, injure, or damage one’s body or kill oneself.|
74+
75+
Classification can be multi-labeled. For example, when a text sample goes through the text moderation model, it could be classified as both Sexual content and Violence.
76+
77+
### Severity levels
78+
79+
Every harm category the service applies also comes with a severity level rating. The severity level is meant to indicate the severity of the consequences of showing the flagged content.
80+
81+
|Severity |Label |
82+
|---------|---------|
83+
|0 |Safe|
84+
|2 |Low|
85+
|4 |Medium|
86+
|6 |High|
87+
88+
### Text blocklist management
89+
90+
Following operations are supported to manage your text blocklist:
91+
92+
* Create or modify a blocklist
93+
* List all blocklists
94+
* Get a blocklist by blocklistName
95+
* Add blockItems to a blocklist
96+
* Remove blockItems from a blocklist
97+
* List all blockItems in a blocklist by blocklistName
98+
* Get a blockItem in a blocklist by blockItemId and blocklistName
99+
* Delete a blocklist and all of its blockItems
100+
101+
You can set the blocklists you want to use when analyze text, then you can get blocklist match result from returned response.
102+
103+
## Examples
104+
The following section provides several code snippets covering some of the most common Content Safety service tasks, including:
105+
106+
* [Analyze text](#analyze-text)
107+
* [Analyze image](#analyze-image)
108+
* [Manage text blocklist](#manage-text-blocklist)
109+
110+
### Analyze text
111+
112+
#### Analyze text without blocklists
113+
```java com.azure.ai.contentsafety.analyzetext
114+
String endpoint = Configuration.getGlobalConfiguration().get("CONTENT_SAFETY_ENDPOINT");
115+
String key = Configuration.getGlobalConfiguration().get("CONTENT_SAFETY_KEY");
116+
ContentSafetyClient contentSafetyClient = new ContentSafetyClientBuilder()
117+
.credential(new KeyCredential(key))
118+
.endpoint(endpoint).buildClient();
119+
120+
AnalyzeTextResult response = contentSafetyClient.analyzeText(new AnalyzeTextOptions("This is text example"));
121+
122+
System.out.println("Hate severity: " + response.getHateResult().getSeverity());
123+
System.out.println("SelfHarm severity: " + response.getSelfHarmResult().getSeverity());
124+
System.out.println("Sexual severity: " + response.getSexualResult().getSeverity());
125+
System.out.println("Violence severity: " + response.getViolenceResult().getSeverity());
126+
```
127+
128+
#### Analyze text with blocklists
129+
```java com.azure.ai.contentsafety.analyzetextwithblocklist
130+
// After you edit your blocklist, it usually takes effect in 5 minutes, please wait some time before analyzing with blocklist after editing.
131+
AnalyzeTextOptions request = new AnalyzeTextOptions("I h*te you and I want to k*ll you");
132+
request.getBlocklistNames().add(blocklistName);
133+
request.setBreakByBlocklists(true);
134+
135+
AnalyzeTextResult analyzeTextResult;
136+
try {
137+
analyzeTextResult = contentSafetyClient.analyzeText(request);
138+
} catch (HttpResponseException ex) {
139+
System.out.println("Analyze text failed.\nStatus code: " + ex.getResponse().getStatusCode() + ", Error message: " + ex.getMessage());
140+
throw ex;
141+
}
142+
143+
if (analyzeTextResult.getBlocklistsMatchResults() != null) {
144+
System.out.println("\nBlocklist match result:");
145+
for (TextBlocklistMatchResult matchResult : analyzeTextResult.getBlocklistsMatchResults()) {
146+
System.out.println("Blockitem was hit in text: Offset: " + matchResult.getOffset() + ", Length: " + matchResult.getLength());
147+
System.out.println("BlocklistName: " + matchResult.getBlocklistName() + ", BlockItemId: " + matchResult.getBlockItemId() + ", BlockItemText: " + matchResult.getBlockItemText());
148+
}
149+
}
150+
```
151+
152+
### Analyze image
153+
```java com.azure.ai.contentsafety.analyzeimage
154+
String endpoint = Configuration.getGlobalConfiguration().get("CONTENT_SAFETY_ENDPOINT");
155+
String key = Configuration.getGlobalConfiguration().get("CONTENT_SAFETY_KEY");
156+
157+
ContentSafetyClient contentSafetyClient = new ContentSafetyClientBuilder()
158+
.credential(new KeyCredential(key))
159+
.endpoint(endpoint).buildClient();
160+
161+
ImageData image = new ImageData();
162+
String cwd = System.getProperty("user.dir");
163+
String source = "/src/samples/resources/image.jpg";
164+
image.setContent(Files.readAllBytes(Paths.get(cwd, source)));
165+
166+
AnalyzeImageResult response =
167+
contentSafetyClient.analyzeImage(new AnalyzeImageOptions(image));
168+
169+
System.out.println("Hate severity: " + response.getHateResult().getSeverity());
170+
System.out.println("SelfHarm severity: " + response.getSelfHarmResult().getSeverity());
171+
System.out.println("Sexual severity: " + response.getSexualResult().getSeverity());
172+
System.out.println("Violence severity: " + response.getViolenceResult().getSeverity());
173+
```
174+
175+
### Manage text blocklist
176+
177+
#### Create or update text blocklist
178+
```java com.azure.ai.contentsafety.createtextblocklist
179+
String blocklistName = "TestBlocklist";
180+
Map<String, String> description = new HashMap<>();
181+
description.put("description", "Test Blocklist");
182+
BinaryData resource = BinaryData.fromObject(description);
183+
RequestOptions requestOptions = new RequestOptions();
184+
Response<BinaryData> response =
185+
contentSafetyClient.createOrUpdateTextBlocklistWithResponse(blocklistName, resource, requestOptions);
186+
if (response.getStatusCode() == 201) {
187+
System.out.println("\nBlocklist " + blocklistName + " created.");
188+
} else if (response.getStatusCode() == 200) {
189+
System.out.println("\nBlocklist " + blocklistName + " updated.");
190+
}
191+
```
192+
#### Add blockItems
193+
```java com.azure.ai.contentsafety.addblockitems
194+
String blockItemText1 = "k*ll";
195+
String blockItemText2 = "h*te";
196+
List<TextBlockItemInfo> blockItems = Arrays.asList(new TextBlockItemInfo(blockItemText1).setDescription("Kill word"),
197+
new TextBlockItemInfo(blockItemText2).setDescription("Hate word"));
198+
AddBlockItemsResult addedBlockItems = contentSafetyClient.addBlockItems(blocklistName, new AddBlockItemsOptions(blockItems));
199+
if (addedBlockItems != null && addedBlockItems.getValue() != null) {
200+
System.out.println("\nBlockItems added:");
201+
for (TextBlockItem addedBlockItem : addedBlockItems.getValue()) {
202+
System.out.println("BlockItemId: " + addedBlockItem.getBlockItemId() + ", Text: " + addedBlockItem.getText() + ", Description: " + addedBlockItem.getDescription());
203+
}
204+
}
205+
```
206+
#### List text blocklists
207+
```java com.azure.ai.contentsafety.listtextblocklists
208+
PagedIterable<TextBlocklist> allTextBlocklists = contentSafetyClient.listTextBlocklists();
209+
System.out.println("\nList Blocklist:");
210+
for (TextBlocklist blocklist : allTextBlocklists) {
211+
System.out.println("Blocklist: " + blocklist.getBlocklistName() + ", Description: " + blocklist.getDescription());
212+
}
213+
```
214+
#### Get text blocklist
215+
```java com.azure.ai.contentsafety.gettextblocklist
216+
TextBlocklist getBlocklist = contentSafetyClient.getTextBlocklist(blocklistName);
217+
if (getBlocklist != null) {
218+
System.out.println("\nGet blocklist:");
219+
System.out.println("BlocklistName: " + getBlocklist.getBlocklistName() + ", Description: " + getBlocklist.getDescription());
220+
}
221+
```
222+
#### List blockItems
223+
``` java com.azure.ai.contentsafety.listtextblocklistitems
224+
PagedIterable<TextBlockItem> allBlockitems = contentSafetyClient.listTextBlocklistItems(blocklistName);
225+
System.out.println("\nList BlockItems:");
226+
for (TextBlockItem blocklistItem : allBlockitems) {
227+
System.out.println("BlockItemId: " + blocklistItem.getBlockItemId() + ", Text: " + blocklistItem.getText() + ", Description: " + blocklistItem.getDescription());
228+
}
229+
```
230+
#### Get blockItem
231+
```java com.azure.ai.contentsafety.gettextblocklistitem
232+
String getBlockItemId = addedBlockItems.getValue().get(0).getBlockItemId();
233+
TextBlockItem getBlockItem = contentSafetyClient.getTextBlocklistItem(blocklistName, getBlockItemId);
234+
System.out.println("\nGet BlockItem:");
235+
System.out.println("BlockItemId: " + getBlockItem.getBlockItemId() + ", Text: " + getBlockItem.getText() + ", Description: " + getBlockItem.getDescription());
236+
```
237+
#### Remove blockItems
238+
```java com.azure.ai.contentsafety.removeblockitems
239+
String removeBlockItemId = addedBlockItems.getValue().get(0).getBlockItemId();
240+
List<String> removeBlockItemIds = new ArrayList<>();
241+
removeBlockItemIds.add(removeBlockItemId);
242+
contentSafetyClient.removeBlockItems(blocklistName, new RemoveBlockItemsOptions(removeBlockItemIds));
243+
```
244+
#### Delete text blocklist
245+
```java com.azure.ai.contentsafety.deletetextblocklist
246+
contentSafetyClient.deleteTextBlocklist(blocklistName);
247+
```
248+
## Troubleshooting
249+
### General
250+
251+
Azure AI Content Safety client library will raise exceptions defined in [Azure Core][azure_core_exception]. Error codes are defined as below:
252+
253+
|Error Code |Possible reasons |Suggestions|
254+
|-----------|-------------------|-----------|
255+
|InvalidRequestBody |One or more fields in the request body do not match the API definition. |1. Check the API version you specified in the API call.<br>2. Check the corresponding API definition for the API version you selected.|
256+
|InvalidResourceName |The resource name you specified in the URL does not meet the requirements, like the blocklist name, blocklist term ID, etc. |1. Check the API version you specified in the API call.<br>2. Check whether the given name has invalid characters according to the API definition.|
257+
|ResourceNotFound |The resource you specified in the URL may not exist, like the blocklist name. |1. Check the API version you specified in the API call.<br>2. Double check the existence of the resource specified in the URL.|
258+
|InternalError |Some unexpected situations on the server side have been triggered. |1. You may want to retry a few times after a small period and see it the issue happens again.<br>2. Contact Azure Support if this issue persists.|
259+
|ServerBusy |The server side cannot process the request temporarily. |1. You may want to retry a few times after a small period and see it the issue happens again.<br>2.Contact Azure Support if this issue persists.|
260+
|TooManyRequests |The current RPS has exceeded the quota for your current SKU. |1. Check the pricing table to understand the RPS quota.<br>2.Contact Azure Support if you need more QPS.|
261+
262+
## Next steps
263+
### Additional documentation
264+
265+
For more extensive documentation on Azure Content Safety, see the [Azure AI Content Safety][contentsafety_overview] on docs.microsoft.com.
266+
267+
## Contributing
268+
269+
For details on contributing to this repository, see the [contributing guide](https://github.com/Azure/azure-sdk-for-java/blob/main/CONTRIBUTING.md).
270+
271+
1. Fork it
272+
1. Create your feature branch (`git checkout -b my-new-feature`)
273+
1. Commit your changes (`git commit -am 'Add some feature'`)
274+
1. Push to the branch (`git push origin my-new-feature`)
275+
1. Create new Pull Request
276+
277+
<!-- LINKS -->
278+
[product_documentation]: https://aka.ms/acs-doc
279+
[docs]: https://azure.github.io/azure-sdk-for-java/
280+
[jdk]: https://docs.microsoft.com/java/azure/jdk/
281+
[azure_subscription]: https://azure.microsoft.com/free/
282+
[azure_identity]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity
283+
[contentsafety_overview]: https://aka.ms/acs-doc
284+
[azure_portal]: https://ms.portal.azure.com/
285+
[azure_cli_endpoint_lookup]: https://docs.microsoft.com/cli/azure/cognitiveservices/account?view=azure-cli-latest#az-cognitiveservices-account-show
286+
[azure_cli_key_lookup]: https://docs.microsoft.com/cli/azure/cognitiveservices/account/keys?view=azure-cli-latest#az-cognitiveservices-account-keys-list
287+
288+
![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-java%2Fsdk%2Fcontentsafety%2Fazure-ai-contentsafety%2FREADME.png)

‎sdk/contentsafety/azure-ai-contentsafety/assets.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"AssetsRepo" : "Azure/azure-sdk-assets",
3+
"AssetsRepoPrefixPath" : "java",
4+
"TagPrefix" : "java/contentsafety/azure-ai-contentsafety",
5+
"Tag" : "java/contentsafety/azure-ai-contentsafety_10dc4f5dac"
6+
}

0 commit comments

Comments
 (0)