Skip to content

Commit 823e5e7

Browse files
authored
Feature/formrecognizerv4 (Azure#24349)
1 parent e782583 commit 823e5e7

File tree

629 files changed

+323537
-14931
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

629 files changed

+323537
-14931
lines changed

sdk/formrecognizer/Azure.AI.FormRecognizer/CHANGELOG.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
# Release History
22

3-
## 3.2.0-beta.1 (Unreleased)
3+
## 4.0.0-beta.1 (Unreleased)
44

55
### Features Added
6-
7-
### Breaking Changes
6+
- This version of the SDK defaults to the latest supported Service API version, which currently is `2021_09_30_preview`.
7+
- Added class `DocumentAnalysisClient` to the new `Azure.AI.FormRecognizer.DocumentAnalysis` namespace. This will be the main client to use when analyzing documents for service versions `2021_09_30_preview` and higher. For lower versions, please use the `FormRecognizerClient`.
8+
- Added methods `StartAnalyzeDocument` and `StartAnalyzeDocumentFromUri` to `DocumentAnalysisClient`. These methods substitute all existing `StartRecognize<...>` methods, such as `StartRecognizeContent` and `StartRecognizeReceiptsFromUri`.
9+
- Added class `DocumentModelAdministrationClient` to the new `Azure.AI.FormRecognizer.DocumentAnalysis` namespace. This will be the main client to use for model management for service versions `2021_09_30_preview` and higher. For lower versions, please use the `FormTrainingClient`.
10+
- Added methods `StartBuildModel`, `StartCopyModel`, `StartCreateComposedModel`, `GetCopyAuthorization`, `GetModel`, `GetModels`, `GetAccountProperties`, `DeleteModel`, `GetOperation`, `GetOperations`, and the equivalent async methods to `DocumentModelAdministrationClient`.
811

912
### Key Bugs Fixed
1013

sdk/formrecognizer/Azure.AI.FormRecognizer/README.md

Lines changed: 15 additions & 372 deletions
Large diffs are not rendered by default.

sdk/formrecognizer/Azure.AI.FormRecognizer/api/Azure.AI.FormRecognizer.netstandard2.0.cs

Lines changed: 409 additions & 0 deletions
Large diffs are not rendered by default.

sdk/formrecognizer/Azure.AI.FormRecognizer/samples/README.md

Lines changed: 15 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -13,29 +13,20 @@ description: Samples for the Azure.AI.FormRecognizer client library
1313
# Azure Form Recognizer client SDK Samples
1414
Azure Cognitive Services Form Recognizer is a cloud service that uses machine learning to recognize form fields, text, and tables in form documents. It includes the following capabilities:
1515

16-
- Recognize form content - Recognize and extract tables, lines, words, and selection marks like radio buttons and check boxes in forms documents, without the need to train a model.
17-
- Recognize custom forms - Recognize and extract form fields and other content from your custom forms, using models you trained with your own form types.
18-
- Recognize Prebuilt models - Recognize data using the following prebuilt models:
19-
- Receipts - Recognize and extract common fields from receipts, using a pre-trained receipt model.
20-
- Business Cards - Recognize and extract common fields from business cards, using a pre-trained business cards model.
21-
- Invoices - Recognize and extract common fields from invoices, using a pre-trained invoice model.
22-
- Identity Documents - Recognize and extract common fields from identity documents like passports or driver's licenses, using a pre-trained identity documents model.
16+
- Layout - Extract text, table structures, and selection marks, along with their bounding region coordinates, from documents.
17+
- Document - Analyze entities, key-value pairs, tables, and selection marks from documents using the general prebuilt document model.
18+
- Prebuilt - Analyze data from certain types of common documents (such as receipts, invoices, business cards, or identity documents) using pre-trained models.
19+
- Custom - Build custom models to analyze text, field values, selection marks, and table data from documents. Custom models are trained with your own data, so they're tailored to your documents.
2320

24-
## Common scenarios samples
25-
- [Recognize form content](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample1_RecognizeFormContent.md)
26-
- [Recognize custom forms](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample2_RecognizeCustomForms.md)
27-
- [Recognize receipts](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample3_RecognizeReceipts.md)
28-
- [Recognize business cards](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample9_RecognizeBusinessCards.md)
29-
- [Recognize invoices](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample10_RecognizeInvoices.md)
30-
- [Recognize identity documents](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample11_RecognizeIdentityDocuments.md)
31-
- [Train a model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample5_TrainModel.md)
32-
- [Manage custom models](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample6_ManageCustomModels.md)
21+
## Common scenarios samples for SDK
22+
- [Extract the layout of a document](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_ExtractLayout.md)
23+
- [Analyze with the prebuilt document model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_AnalyzePrebuiltDocument.md)
24+
- [Analyze a document with a custom model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_AnalyzeWithCustomModel.md)
25+
- [Analyze a document with a prebuilt model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_AnalyzeWithPrebuiltModel.md)
26+
- [Build a model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_BuildModel.md)
27+
- [Manage models](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_ManageModels.md)
3328

34-
## Advanced samples
35-
- [Strongly-typing a recognized form](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample4_StronglyTypingARecognizedForm.md)
36-
- [Create a composed model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample8_ModelCompose.md)
37-
- [Differentiate output models trained with and without labels](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/tests/samples/Sample10_DifferentiateOutputModelsTrainedWithAndWithoutLabels.cs)
38-
- [Differentiate output labeled tables](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/tests/samples/Sample15_DifferentiateOutputLabeledTables.cs)
39-
- [Copy a custom model between Form Recognizer resources](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample7_CopyCustomModel.md)
40-
- [Field Bounding Box](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/tests/samples/Sample9_FieldBoundingBox.cs)
41-
- [Mock a client for testing using the Moq library](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_MockClient.md)
29+
## Advanced samples for SDK
30+
- [Create a composed model](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_ModelCompose.md)
31+
- [Copy a custom model between Form Recognizer resources](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_CopyCustomModel.md)
32+
- [Get and List document model operations](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer/samples/Sample_GetAndListOperations.md)
Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
# Analyze with the prebuilt document model
2+
3+
This sample demonstrates how to analyze entities, key-value pairs, tables, and selection marks from documents using the general prebuilt document model.
4+
5+
To get started you'll need a Cognitive Services resource or a Form Recognizer resource. See [README][README] for prerequisites and instructions.
6+
7+
## Creating a `DocumentAnalysisClient`
8+
9+
To create a new `DocumentAnalysisClient` you need the endpoint and credentials from your resource. In the sample below you'll use a Form Recognizer API key credential by creating an `AzureKeyCredential` object, that if needed, will allow you to update the API key without creating a new client.
10+
11+
You can set `endpoint` and `apiKey` based on an environment variable, a configuration setting, or any way that works for your application.
12+
13+
```C# Snippet:CreateDocumentAnalysisClient
14+
string endpoint = "<endpoint>";
15+
string apiKey = "<apiKey>";
16+
var credential = new AzureKeyCredential(apiKey);
17+
var client = new DocumentAnalysisClient(new Uri(endpoint), credential);
18+
```
19+
20+
## Use the prebuilt document model to analyze a document from a URI
21+
22+
To analyze a given file at a URI, use the `StartAnalyzeDocumentFromUri` method and pass `prebuilt-document` as the model ID. The returned value is an `AnalyzeResult` object containing data about the submitted document.
23+
24+
```C# Snippet:FormRecognizerAnalyzePrebuiltDocumentFromUriAsync
25+
string fileUri = "<fileUri>";
26+
27+
AnalyzeDocumentOperation operation = await client.StartAnalyzeDocumentFromUriAsync("prebuilt-document", fileUri);
28+
29+
await operation.WaitForCompletionAsync();
30+
31+
AnalyzeResult result = operation.Value;
32+
33+
foreach (DocumentPage page in result.Pages)
34+
{
35+
Console.WriteLine($"Document Page {page.PageNumber} has {page.Lines.Count} line(s), {page.Words.Count} word(s),");
36+
Console.WriteLine($"and {page.SelectionMarks.Count} selection mark(s).");
37+
38+
for (int i = 0; i < page.Lines.Count; i++)
39+
{
40+
DocumentLine line = page.Lines[i];
41+
Console.WriteLine($" Line {i} has content: '{line.Content}'.");
42+
43+
Console.WriteLine($" Its bounding box is:");
44+
Console.WriteLine($" Upper left => X: {line.BoundingBox[0].X}, Y= {line.BoundingBox[0].Y}");
45+
Console.WriteLine($" Upper right => X: {line.BoundingBox[1].X}, Y= {line.BoundingBox[1].Y}");
46+
Console.WriteLine($" Lower right => X: {line.BoundingBox[2].X}, Y= {line.BoundingBox[2].Y}");
47+
Console.WriteLine($" Lower left => X: {line.BoundingBox[3].X}, Y= {line.BoundingBox[3].Y}");
48+
}
49+
50+
for (int i = 0; i < page.SelectionMarks.Count; i++)
51+
{
52+
DocumentSelectionMark selectionMark = page.SelectionMarks[i];
53+
54+
Console.WriteLine($" Selection Mark {i} is {selectionMark.State}.");
55+
Console.WriteLine($" Its bounding box is:");
56+
Console.WriteLine($" Upper left => X: {selectionMark.BoundingBox[0].X}, Y= {selectionMark.BoundingBox[0].Y}");
57+
Console.WriteLine($" Upper right => X: {selectionMark.BoundingBox[1].X}, Y= {selectionMark.BoundingBox[1].Y}");
58+
Console.WriteLine($" Lower right => X: {selectionMark.BoundingBox[2].X}, Y= {selectionMark.BoundingBox[2].Y}");
59+
Console.WriteLine($" Lower left => X: {selectionMark.BoundingBox[3].X}, Y= {selectionMark.BoundingBox[3].Y}");
60+
}
61+
}
62+
63+
foreach (DocumentStyle style in result.Styles)
64+
{
65+
// Check the style and style confidence to see if text is handwritten.
66+
// Note that value '0.8' is used as an example.
67+
68+
bool isHandwritten = style.IsHandwritten.HasValue && style.IsHandwritten == true;
69+
70+
if (isHandwritten && style.Confidence > 0.8)
71+
{
72+
Console.WriteLine($"Handwritten content found:");
73+
74+
foreach (DocumentSpan span in style.Spans)
75+
{
76+
Console.WriteLine($" Content: {result.Content.Substring(span.Offset, span.Length)}");
77+
}
78+
}
79+
}
80+
81+
Console.WriteLine("The following tables were extracted:");
82+
83+
for (int i = 0; i < result.Tables.Count; i++)
84+
{
85+
DocumentTable table = result.Tables[i];
86+
Console.WriteLine($" Table {i} has {table.RowCount} rows and {table.ColumnCount} columns.");
87+
88+
foreach (DocumentTableCell cell in table.Cells)
89+
{
90+
Console.WriteLine($" Cell ({cell.RowIndex}, {cell.ColumnIndex}) has kind '{cell.Kind}' and content: '{cell.Content}'.");
91+
}
92+
}
93+
94+
Console.WriteLine("Detected entities:");
95+
96+
foreach (DocumentEntity entity in result.Entities)
97+
{
98+
if (entity.SubCategory == null)
99+
{
100+
Console.WriteLine($" Found entity '{entity.Content}' with category '{entity.Category}'.");
101+
}
102+
else
103+
{
104+
Console.WriteLine($" Found entity '{entity.Content}' with category '{entity.Category}' and sub-category '{entity.SubCategory}'.");
105+
}
106+
}
107+
108+
Console.WriteLine("Detected key-value pairs:");
109+
110+
foreach (DocumentKeyValuePair kvp in result.KeyValuePairs)
111+
{
112+
if (kvp.Value.Content == null)
113+
{
114+
Console.WriteLine($" Found key with no value: '{kvp.Key.Content}'");
115+
}
116+
else
117+
{
118+
Console.WriteLine($" Found key-value pair: '{kvp.Key.Content}' and '{kvp.Value.Content}'");
119+
}
120+
}
121+
```
122+
123+
## Use the prebuilt document model to analyze a document from a file stream
124+
125+
To analyze a given file at a file stream, use the `StartAnalyzeDocument` method and pass `prebuilt-document` as the model ID. The returned value is an `AnalyzeResult` object containing data about the submitted document.
126+
127+
```C# Snippet:FormRecognizerAnalyzePrebuiltDocumentFromFileAsync
128+
string filePath = "filePath";
129+
using var stream = new FileStream(filePath, FileMode.Open);
130+
131+
AnalyzeDocumentOperation operation = await client.StartAnalyzeDocumentAsync("prebuilt-document", stream);
132+
133+
await operation.WaitForCompletionAsync();
134+
135+
AnalyzeResult result = operation.Value;
136+
137+
foreach (DocumentPage page in result.Pages)
138+
{
139+
Console.WriteLine($"Document Page {page.PageNumber} has {page.Lines.Count} line(s), {page.Words.Count} word(s),");
140+
Console.WriteLine($"and {page.SelectionMarks.Count} selection mark(s).");
141+
142+
for (int i = 0; i < page.Lines.Count; i++)
143+
{
144+
DocumentLine line = page.Lines[i];
145+
Console.WriteLine($" Line {i} has content: '{line.Content}'.");
146+
147+
Console.WriteLine($" Its bounding box is:");
148+
Console.WriteLine($" Upper left => X: {line.BoundingBox[0].X}, Y= {line.BoundingBox[0].Y}");
149+
Console.WriteLine($" Upper right => X: {line.BoundingBox[1].X}, Y= {line.BoundingBox[1].Y}");
150+
Console.WriteLine($" Lower right => X: {line.BoundingBox[2].X}, Y= {line.BoundingBox[2].Y}");
151+
Console.WriteLine($" Lower left => X: {line.BoundingBox[3].X}, Y= {line.BoundingBox[3].Y}");
152+
}
153+
154+
for (int i = 0; i < page.SelectionMarks.Count; i++)
155+
{
156+
DocumentSelectionMark selectionMark = page.SelectionMarks[i];
157+
158+
Console.WriteLine($" Selection Mark {i} is {selectionMark.State}.");
159+
Console.WriteLine($" Its bounding box is:");
160+
Console.WriteLine($" Upper left => X: {selectionMark.BoundingBox[0].X}, Y= {selectionMark.BoundingBox[0].Y}");
161+
Console.WriteLine($" Upper right => X: {selectionMark.BoundingBox[1].X}, Y= {selectionMark.BoundingBox[1].Y}");
162+
Console.WriteLine($" Lower right => X: {selectionMark.BoundingBox[2].X}, Y= {selectionMark.BoundingBox[2].Y}");
163+
Console.WriteLine($" Lower left => X: {selectionMark.BoundingBox[3].X}, Y= {selectionMark.BoundingBox[3].Y}");
164+
}
165+
}
166+
167+
foreach (DocumentStyle style in result.Styles)
168+
{
169+
// Check the style and style confidence to see if text is handwritten.
170+
// Note that value '0.8' is used as an example.
171+
172+
bool isHandwritten = style.IsHandwritten.HasValue && style.IsHandwritten == true;
173+
174+
if (isHandwritten && style.Confidence > 0.8)
175+
{
176+
Console.WriteLine($"Handwritten content found:");
177+
178+
foreach (DocumentSpan span in style.Spans)
179+
{
180+
Console.WriteLine($" Content: {result.Content.Substring(span.Offset, span.Length)}");
181+
}
182+
}
183+
}
184+
185+
Console.WriteLine("The following tables were extracted:");
186+
187+
for (int i = 0; i < result.Tables.Count; i++)
188+
{
189+
DocumentTable table = result.Tables[i];
190+
Console.WriteLine($" Table {i} has {table.RowCount} rows and {table.ColumnCount} columns.");
191+
192+
foreach (DocumentTableCell cell in table.Cells)
193+
{
194+
Console.WriteLine($" Cell ({cell.RowIndex}, {cell.ColumnIndex}) has kind '{cell.Kind}' and content: '{cell.Content}'.");
195+
}
196+
}
197+
198+
Console.WriteLine("Detected entities:");
199+
200+
foreach (DocumentEntity entity in result.Entities)
201+
{
202+
if (entity.SubCategory == null)
203+
{
204+
Console.WriteLine($" Found entity '{entity.Content}' with category '{entity.Category}'.");
205+
}
206+
else
207+
{
208+
Console.WriteLine($" Found entity '{entity.Content}' with category '{entity.Category}' and sub-category '{entity.SubCategory}'.");
209+
}
210+
}
211+
212+
Console.WriteLine("Detected key-value pairs:");
213+
214+
foreach (DocumentKeyValuePair kvp in result.KeyValuePairs)
215+
{
216+
if (kvp.Value.Content == null)
217+
{
218+
Console.WriteLine($" Found key with no value: '{kvp.Key.Content}'");
219+
}
220+
else
221+
{
222+
Console.WriteLine($" Found key-value pair: '{kvp.Key.Content}' and '{kvp.Value.Content}'");
223+
}
224+
}
225+
```
226+
227+
To see the full example source files, see:
228+
229+
* [Analyze with prebuilt document from URI](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/tests/samples/Sample_AnalyzePrebuiltDocumentFromUriAsync.cs)
230+
* [Analyze with prebuilt document from file](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/formrecognizer/Azure.AI.FormRecognizer/tests/samples/Sample_AnalyzePrebuiltDocumentFromFileAsync.cs)
231+
232+
[README]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/formrecognizer/Azure.AI.FormRecognizer#getting-started

0 commit comments

Comments
 (0)