Skip to content

Commit fc8a5fc

Browse files
authored
[Search] Regenerate code using 2023-10-01-preview API version (Azure#38925)
* Regenerate code
1 parent fba9f87 commit fc8a5fc

File tree

109 files changed

+4690
-831
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

109 files changed

+4690
-831
lines changed

.vscode/cspell.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,9 @@
114114
"unhold",
115115
"uninstrumented",
116116
"vcpus",
117+
"Vectorizable",
118+
"Vectorizer",
119+
"Vectorizers",
117120
"vmss",
118121
"vnet",
119122
"westcentralus",

sdk/search/Azure.Search.Documents/api/Azure.Search.Documents.netstandard2.0.cs

Lines changed: 212 additions & 15 deletions
Large diffs are not rendered by default.

sdk/search/Azure.Search.Documents/assets.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22
"AssetsRepo": "Azure/azure-sdk-assets",
33
"AssetsRepoPrefixPath": "net",
44
"TagPrefix": "net/search/Azure.Search.Documents",
5-
"Tag": "net/search/Azure.Search.Documents_38f84358e8"
5+
"Tag": "net/search/Azure.Search.Documents_53888408bf"
66
}

sdk/search/Azure.Search.Documents/samples/Sample07_VectorSearch.md

Lines changed: 16 additions & 344 deletions
Large diffs are not rendered by default.
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Vector Search Using Field Builder
2+
3+
The `FieldBuilder` class allows you to define a Search index from a model type. This sample demonstrates how to create a vector fields index using the field builder.
4+
5+
## Model Creation
6+
7+
Consider the following model, which includes a property named `DescriptionVector` that represents a vector field. To configure a vector field, you must provide the model dimensions, indicating the size of the embeddings generated for this field, as well as the name of the vector search profile that specifies the algorithm configuration and any optional parameters for searching the vector field.
8+
9+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_FieldBuilder_Model
10+
public class MyDocument
11+
{
12+
[SimpleField(IsKey = true, IsFilterable = true, IsSortable = true)]
13+
public string Id { get; set; }
14+
15+
[SearchableField(IsFilterable = true, IsSortable = true)]
16+
public string Name { get; set; }
17+
18+
[SearchableField(AnalyzerName = "en.microsoft")]
19+
public string Description { get; set; }
20+
21+
[SearchableField(VectorSearchDimensions = "1536", VectorSearchProfile = "my-vector-profile")]
22+
public IReadOnlyList<float> DescriptionVector { get; set; }
23+
}
24+
```
25+
26+
## Create an Index using `FieldBuilder`
27+
28+
We will create an instace of `SearchIndex` and use `FieldBuilder` to define fields based on the `MyDocument` model class.
29+
30+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Index_UsingFieldBuilder
31+
string vectorSearchProfile = "my-vector-profile";
32+
string vectorSearchHnswConfig = "my-hsnw-vector-config";
33+
34+
string indexName = "MyDocument";
35+
// Create Index
36+
SearchIndex searchIndex = new SearchIndex(indexName)
37+
{
38+
Fields = new FieldBuilder().Build(typeof(MyDocument)),
39+
VectorSearch = new()
40+
{
41+
Profiles =
42+
{
43+
new VectorSearchProfile(vectorSearchProfile, vectorSearchHnswConfig)
44+
},
45+
Algorithms =
46+
{
47+
new HnswVectorSearchAlgorithmConfiguration(vectorSearchHnswConfig)
48+
}
49+
},
50+
};
51+
```
52+
53+
After creating an instance of the `SearchIndex`, we need to instantiate the `SearchIndexClient` and call the `CreateIndex` method to create the search index.
54+
55+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Create_Index_FieldBuilder
56+
Uri endpoint = new(Environment.GetEnvironmentVariable("SEARCH_ENDPOINT"));
57+
string key = Environment.GetEnvironmentVariable("SEARCH_API_KEY");
58+
AzureKeyCredential credential = new(key);
59+
60+
SearchIndexClient indexClient = new(endpoint, credential);
61+
await indexClient.CreateIndexAsync(searchIndex);
62+
```
63+
64+
To perform vector search please refer to the [Vector Search Using RAW Vector Query](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample07_VectorSearch_UsingRawVectorQuery.md) or [Vector Search Using Vectorizable Text Query](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample07_VectorSearch_UsingVectorizableTextQuery.md) samples.
Lines changed: 272 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,272 @@
1+
# Vector Search Using RAW Vector Query
2+
3+
This sample demonstrates how to create a vector fields index, upload data into the index, and perform various types of vector searches using raw vector queries.
4+
5+
## Create a Vector Index
6+
7+
Let's consider the example of a `Hotel`. First, we need to create an index for storing hotel information. In this index, we will define vector fields called `DescriptionVector` and `CategoryVector`. To configure the vector field, you need to provide the model dimensions, which indicate the size of the embeddings generated for this field, and the name of the vector search profile that specifies the algorithm configuration and any optional parameters for searching the vector field. You can find detailed instructions on how to create a vector index in the [documentation](https://learn.microsoft.com/azure/search/vector-search-how-to-create-index).
8+
9+
We will create an instace of `SearchIndex` and define `Hotel` fields.
10+
11+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Index_UsingRawVectors
12+
string vectorSearchProfile = "my-vector-profile";
13+
string vectorSearchHnswConfig = "my-hsnw-vector-config";
14+
int modelDimensions = 1536;
15+
16+
string indexName = "Hotel";
17+
SearchIndex searchIndex = new(indexName)
18+
{
19+
Fields =
20+
{
21+
new SimpleField("HotelId", SearchFieldDataType.String) { IsKey = true, IsFilterable = true, IsSortable = true, IsFacetable = true },
22+
new SearchableField("HotelName") { IsFilterable = true, IsSortable = true },
23+
new SearchableField("Description") { IsFilterable = true },
24+
new SearchField("DescriptionVector", SearchFieldDataType.Collection(SearchFieldDataType.Single))
25+
{
26+
IsSearchable = true,
27+
VectorSearchDimensions = modelDimensions,
28+
VectorSearchProfile = vectorSearchProfile
29+
},
30+
new SearchableField("Category") { IsFilterable = true, IsSortable = true, IsFacetable = true },
31+
new SearchField("CategoryVector", SearchFieldDataType.Collection(SearchFieldDataType.Single))
32+
{
33+
IsSearchable = true,
34+
VectorSearchDimensions = modelDimensions,
35+
VectorSearchProfile = vectorSearchProfile
36+
},
37+
},
38+
VectorSearch = new()
39+
{
40+
Profiles =
41+
{
42+
new VectorSearchProfile(vectorSearchProfile, vectorSearchHnswConfig)
43+
},
44+
Algorithms =
45+
{
46+
new HnswVectorSearchAlgorithmConfiguration(vectorSearchHnswConfig)
47+
}
48+
},
49+
};
50+
```
51+
52+
After creating an instance of the `SearchIndex`, we need to instantiate the `SearchIndexClient` and call the `CreateIndex` method to create the search index.
53+
54+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Create_Index_UsingRawVectors
55+
Uri endpoint = new(Environment.GetEnvironmentVariable("SEARCH_ENDPOINT"));
56+
string key = Environment.GetEnvironmentVariable("SEARCH_API_KEY");
57+
AzureKeyCredential credential = new(key);
58+
59+
SearchIndexClient indexClient = new(endpoint, credential);
60+
await indexClient.CreateIndexAsync(searchIndex);
61+
```
62+
63+
## Add documents to your index
64+
65+
Let's create a simple model type for `Hotel`:
66+
67+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Model
68+
public class Hotel
69+
{
70+
public string HotelId { get; set; }
71+
public string HotelName { get; set; }
72+
public string Description { get; set; }
73+
public IReadOnlyList<float> DescriptionVector { get; set; }
74+
public string Category { get; set; }
75+
public IReadOnlyList<float> CategoryVector { get; set; }
76+
}
77+
```
78+
79+
Next, we will create sample hotel documents. The vector field requires submitting text input to an embedding model that converts human-readable text into a vector representation. To convert a text query string provided by a user into a vector representation, your application should utilize an embedding library that offers this functionality. For more details about how to generate embeddings, refer to the [documentation](https://learn.microsoft.com/azure/search/vector-search-how-to-generate-embeddings). Here's an example of how you can get embeddings using [Azure.AI.OpenAI](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/openai/Azure.AI.OpenAI/README.md) library.
80+
81+
### Get Embeddings using `Azure.AI.OpenAI`
82+
83+
```C# Snippet:Azure_Search_Tests_Samples_Readme_GetEmbeddings
84+
Uri endpoint = new Uri(Environment.GetEnvironmentVariable("OpenAI_ENDPOINT"));
85+
string key = Environment.GetEnvironmentVariable("OpenAI_API_KEY");
86+
AzureKeyCredential credential = new AzureKeyCredential(key);
87+
88+
OpenAIClient openAIClient = new OpenAIClient(endpoint, credential);
89+
string description = "Very popular hotel in town.";
90+
EmbeddingsOptions embeddingsOptions = new(description);
91+
92+
Embeddings embeddings = await openAIClient.GetEmbeddingsAsync("EmbeddingsModelName", embeddingsOptions);
93+
IReadOnlyList<float> descriptionVector = embeddings.Data[0].Embedding;
94+
```
95+
96+
In the sample code below, we are using hardcoded embeddings for the vector fields named `DescriptionVector` and `CategoryVector`:
97+
98+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Hotel_Document
99+
public static Hotel[] GetHotelDocuments()
100+
{
101+
return new[]
102+
{
103+
new Hotel()
104+
{
105+
HotelId = "1",
106+
HotelName = "Fancy Stay",
107+
Description =
108+
"Best hotel in town if you like luxury hotels. They have an amazing infinity pool, a spa, " +
109+
"and a really helpful concierge. The location is perfect -- right downtown, close to all " +
110+
"the tourist attractions. We highly recommend this hotel.",
111+
DescriptionVector = VectorSearchEmbeddings.Hotel1VectorizeDescription,
112+
Category = "Luxury",
113+
CategoryVector = VectorSearchEmbeddings.LuxuryVectorizeCategory
114+
},
115+
new Hotel()
116+
{
117+
HotelId = "2",
118+
HotelName = "Roach Motel",
119+
Description = "Cheapest hotel in town. Infact, a motel.",
120+
DescriptionVector = VectorSearchEmbeddings.Hotel2VectorizeDescription,
121+
Category = "Budget",
122+
CategoryVector = VectorSearchEmbeddings.BudgetVectorizeCategory
123+
},
124+
// Add more hotel documents here...
125+
};
126+
}
127+
```
128+
129+
Now, we can instantiate the `SearchClient` and upload the documents to the `Hotel` index we created earlier:
130+
131+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Upload_Documents_UsingRawVectors
132+
SearchClient searchClient = new(endpoint, indexName, credential);
133+
Hotel[] hotelDocuments = GetHotelDocuments();
134+
await searchClient.IndexDocumentsAsync(IndexDocumentsBatch.Upload(hotelDocuments));
135+
```
136+
137+
## Query Vector Data
138+
139+
When using `RawVectorQuery`, the query for a vector field must also be a vector. To convert a text query string provided by a user into a vector representation, your application must call an embedding library that provides this capability. Use the same embedding library that you used to generate embeddings in the source documents. For more details on how to generate embeddings, please refer to the [documentation](https://learn.microsoft.com/azure/search/vector-search-how-to-generate-embeddings). In the sample codes below, we are using hardcoded embeddings to query vector field.
140+
141+
Let's query the index and make sure everything works as implemented. You can also refer to the [documentation](https://learn.microsoft.com/azure/search/vector-search-how-to-query) for more information on querying vector data.
142+
143+
### Single Vector Search
144+
145+
In this vector query, the `VectorQueries` collection contains the vectors representing the query input. The `Fields` property specifies which vector fields are searched. The `KNearestNeighborsCount` property specifies the number of nearest neighbors to return as top hits.
146+
147+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Single_Vector_Search_UsingRawVectors
148+
IReadOnlyList<float> vectorizedResult = VectorSearchEmbeddings.SearchVectorizeDescription; // "Top hotels in town"
149+
150+
SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(null,
151+
new SearchOptions
152+
{
153+
VectorQueries = { new RawVectorQuery() { Vector = vectorizedResult, KNearestNeighborsCount = 3, Fields = { "DescriptionVector" } } },
154+
});
155+
156+
int count = 0;
157+
Console.WriteLine($"Single Vector Search Results:");
158+
await foreach (SearchResult<Hotel> result in response.GetResultsAsync())
159+
{
160+
count++;
161+
Hotel doc = result.Document;
162+
Console.WriteLine($"{doc.HotelId}: {doc.HotelName}");
163+
}
164+
Console.WriteLine($"Total number of search results:{count}");
165+
```
166+
167+
### Single Vector Search With Filter
168+
169+
In addition to the vector query mentioned above, we can also apply a filter to narrow down the search results.
170+
171+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Filter_UsingRawVectors
172+
IReadOnlyList<float> vectorizedResult = VectorSearchEmbeddings.SearchVectorizeDescription; // "Top hotels in town"
173+
174+
SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(null,
175+
new SearchOptions
176+
{
177+
VectorQueries = { new RawVectorQuery() { Vector = vectorizedResult, KNearestNeighborsCount = 3, Fields = { "DescriptionVector" } } },
178+
Filter = "Category eq 'Luxury'"
179+
});
180+
181+
int count = 0;
182+
Console.WriteLine($"Single Vector Search With Filter Results:");
183+
await foreach (SearchResult<Hotel> result in response.GetResultsAsync())
184+
{
185+
count++;
186+
Hotel doc = result.Document;
187+
Console.WriteLine($"{doc.HotelId}: {doc.HotelName}");
188+
}
189+
Console.WriteLine($"Total number of search results:{count}");
190+
```
191+
192+
### Hybrid Search
193+
194+
A hybrid query combines full text search, semantic search (reranking), and vector search. The search engine runs full text and vector queries in parallel. Semantic ranking is applied to the results from the text search. A single result set is returned in the response.
195+
196+
#### Simple Hybrid Search
197+
198+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Simple_Hybrid_Search_UsingRawVectors
199+
IReadOnlyList<float> vectorizedResult = VectorSearchEmbeddings.SearchVectorizeDescription; // "Top hotels in town"
200+
201+
SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(
202+
"Top hotels in town",
203+
new SearchOptions
204+
{
205+
VectorQueries = { new RawVectorQuery() { Vector = vectorizedResult, KNearestNeighborsCount = 3, Fields = { "DescriptionVector" } } },
206+
});
207+
208+
int count = 0;
209+
Console.WriteLine($"Simple Hybrid Search Results:");
210+
await foreach (SearchResult<Hotel> result in response.GetResultsAsync())
211+
{
212+
count++;
213+
Hotel doc = result.Document;
214+
Console.WriteLine($"{doc.HotelId}: {doc.HotelName}");
215+
}
216+
Console.WriteLine($"Total number of search results:{count}");
217+
```
218+
219+
### Multi-vector Search
220+
221+
You can search containing multiple query vectors using the `SearchOptions.VectorQueries` property. These queries will be executed concurrently in the search index, with each one searching for similarities in the target vector fields. The result set will be a combination of documents that matched both vector queries. One common use case for this query request is when using models like CLIP for a multi-modal vector search, where the same model can vectorize both image and non-image content.
222+
223+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Multi_Vector_Search_UsingRawVectors
224+
IReadOnlyList<float> vectorizedDescriptionQuery = VectorSearchEmbeddings.SearchVectorizeDescription; // "Top hotels in town"
225+
IReadOnlyList<float> vectorizedCategoryQuery = VectorSearchEmbeddings.SearchVectorizeCategory; // "Luxury hotels in town"
226+
227+
SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(null,
228+
new SearchOptions
229+
{
230+
VectorQueries = {
231+
new RawVectorQuery() { Vector = vectorizedDescriptionQuery, KNearestNeighborsCount = 3, Fields = { "DescriptionVector" } },
232+
new RawVectorQuery() { Vector = vectorizedCategoryQuery, KNearestNeighborsCount = 3, Fields = { "CategoryVector" } }
233+
},
234+
});
235+
236+
int count = 0;
237+
Console.WriteLine($"Multi Vector Search Results:");
238+
await foreach (SearchResult<Hotel> result in response.GetResultsAsync())
239+
{
240+
count++;
241+
Hotel doc = result.Document;
242+
Console.WriteLine($"{doc.HotelId}: {doc.HotelName}");
243+
}
244+
Console.WriteLine($"Total number of search results:{count}");
245+
```
246+
247+
### Multi-field Vector Search
248+
249+
You can set the `SearchOptions.VectorQueries.Fields` property to multiple vector fields. For example, we have vector fields named `DescriptionVector` and `CategoryVector`. Your vector query executes over both the `DescriptionVector` and `CategoryVector` fields, which must have the same embedding space since they share the same query vector.
250+
251+
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Multi_Fields_Vector_Search_UsingRawVectors
252+
IReadOnlyList<float> vectorizedResult = VectorSearchEmbeddings.SearchVectorizeDescription; // "Top hotels in town"
253+
254+
SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(null,
255+
new SearchOptions
256+
{
257+
VectorQueries = { new RawVectorQuery() {
258+
Vector = vectorizedResult,
259+
KNearestNeighborsCount = 3,
260+
Fields = { "DescriptionVector", "CategoryVector" } } }
261+
});
262+
263+
int count = 0;
264+
Console.WriteLine($"Multi Fields Vector Search Results:");
265+
await foreach (SearchResult<Hotel> result in response.GetResultsAsync())
266+
{
267+
count++;
268+
Hotel doc = result.Document;
269+
Console.WriteLine($"{doc.HotelId}: {doc.HotelName}");
270+
}
271+
Console.WriteLine($"Total number of search results:{count}");
272+
```

0 commit comments

Comments
 (0)