Skip to content

Commit cc8b418

Browse files
Blobs migration additions (Azure#20965)
* blob metadata section * Test new migration samples * sample link fix * pr feedback Co-authored-by: jschrepp-MSFT <41338290+jschrepp-MSFT@users.noreply.github.com>
1 parent 5f16365 commit cc8b418

File tree

2 files changed

+137
-2
lines changed

2 files changed

+137
-2
lines changed

sdk/storage/Azure.Storage.Blobs/AzureStorageNetMigrationV12.md

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ Familiarity with the legacy client library is assumed. For those new to the Azur
2222
- [Uploading Blobs to a Container](#uploading-blobs-to-a-container)
2323
- [Downloading Blobs from a Container](#downloading-blobs-from-a-container)
2424
- [Listing Blobs in a Container](#listing-blobs-in-a-container)
25+
- [Managing Blob Metadata](#managing-blob-metadata)
2526
- [Generate a SAS](#generate-a-sas)
2627
- [Content Hashes](#content-hashes)
2728
- [Resiliency](#resiliency)
@@ -194,9 +195,9 @@ await containerClient.SetAccessPolicyAsync(permissions: signedIdentifiers);
194195

195196
### Client Structure
196197

197-
The legacy SDK used a stateful model. There were container and blob objects that held state regarding service resources and required the user to manually call their update methods. But blob contents were not a part of this state and had to be uploaded/downloaded whenever they were to be interacted with. This became increasingly confusing over time, and increasingly susceptible to thread safety issues.
198+
**The legacy SDK used a stateful model.** There were container and blob objects that held state regarding service resources and required the user to manually call their update methods. But blob contents were not a part of this state and had to be uploaded/downloaded whenever they were to be interacted with. This became increasingly confusing over time, and increasingly susceptible to thread safety issues.
198199

199-
The modern SDK has taken a client-based approach. There are no objects designed to be representations of storage resources, but instead clients that act as your mechanism to interact with your storage resources in the cloud. Clients hold no state of your resources.
200+
The modern SDK has taken a client-based approach. There are no objects designed to be representations of storage resources, but instead clients that act as your mechanism to interact with your storage resources in the cloud. **Clients hold no state of your resources.** This is most noticable when looking at [blob metadata](#managing-blob-metadata).
200201

201202
The hierarchical structure of Azure Blob Storage can be understood by the following diagram:
202203
![Blob Storage Hierarchy](https://docs.microsoft.com/en-us/azure/storage/blobs/media/storage-blobs-introduction/blob1.png)
@@ -452,6 +453,61 @@ await foreach (BlobHierarchyItem item in results)
452453
}
453454
```
454455

456+
### Managing Blob Metadata
457+
458+
On the service, blob metadata is overwritten alongside blob data overwrites. If metadata is not provided on a blob content edit, that is interpreted as a metadata clear. Legacy versions of the SDK mitigated this by maintaining blob metadata internally and sending it for you on appropriate requests. This helped in simple cases, but could fall out of sync and required developers to defensively code against metadata changes in a multi-client scenario anyway.
459+
460+
V12 has abandoned this stateful approach, having users manage their own metadata. While this requires additional code for developers, it ensures you always know how your metadata is being managed and avoid silently corrupting metadata due to SDK caching.
461+
462+
v11 samples:
463+
464+
The legacy SDK maintained a metadata cache, allowing you to modify metadata on the CloudBlob and invoke Update(). Calling FetchAttributes beforehand refreshed the metadata cache to avoid undoing recent changes.
465+
466+
```csharp
467+
cloudBlob.FetchAttributes();
468+
cloudBlob.Metadata.Add("foo", "bar");
469+
cloudBlob.SetMetadata(metadata);
470+
```
471+
472+
The legacy SDK maintained internal state for blob content uploads. Calling FetchAttributes beforehand refreshed the metadata cache to avoid undoing recent changes.
473+
474+
```csharp
475+
// download blob content. blob metadata is fetched and cached on download
476+
cloudBlob.DownloadToByteArray(downloadBuffer, 0);
477+
478+
// modify blob content
479+
string modifiedBlobContent = Encoding.UTF8.GetString(downloadBuffer) + "FizzBuzz";
480+
481+
// reupload modified blob content while preserving metadata
482+
blobClient.UploadText(modifiedBlobContent);
483+
```
484+
485+
v12 samples:
486+
487+
The modern SDK requires you to hold onto metadata and update it approprately before sending off. You cannot just add a new key-value pair, you must update the collection and send the collection.
488+
489+
```C# Snippet:SampleSnippetsBlobMigration_EditMetadata
490+
IDictionary<string, string> metadata = blobClient.GetProperties().Value.Metadata;
491+
metadata.Add("foo", "bar");
492+
blobClient.SetMetadata(metadata);
493+
```
494+
495+
Additionally with blob content edits, if your blobs have metadata you need to get the metadata and reupload with that metadata, telling the service what metadata goes with this new blob state.
496+
497+
```C# Snippet:SampleSnippetsBlobMigration_EditBlobWithMetadata
498+
// download blob content and metadata
499+
BlobDownloadResult blobData = blobClient.DownloadContent();
500+
501+
// modify blob content
502+
string modifiedBlobContent = blobData.Content + "FizzBuzz";
503+
504+
// reupload modified blob content while preserving metadata
505+
// not adding metadata is a metadata clear
506+
blobClient.Upload(
507+
BinaryData.FromString(modifiedBlobContent),
508+
new BlobUploadOptions() { Metadata = blobData.Details.Metadata });
509+
```
510+
455511
### Generate a SAS
456512

457513
There are various SAS tokens that may be generated. Visit our documentation pages to learn more: [Create a User Delegation SAS](https://docs.microsoft.com/azure/storage/blobs/storage-blob-user-delegation-sas-create-dotnet), [Create a Service SAS](https://docs.microsoft.com/azure/storage/blobs/storage-blob-service-sas-create-dotnet), or [Create an Account SAS](https://docs.microsoft.com/azure/storage/common/storage-account-sas-create-dotnet?toc=/azure/storage/blobs/toc.json).

sdk/storage/Azure.Storage.Blobs/samples/Sample03_Migrations.cs

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -602,6 +602,85 @@ void MyConsumeBlobItemFunc(BlobHierarchyItem item)
602602
}
603603
}
604604

605+
[Test]
606+
public async Task EditMetadata()
607+
{
608+
string data = "hello world";
609+
var initialMetadata = new Dictionary<string, string> { { "fizz", "buzz" } };
610+
611+
string containerName = Randomize("sample-container");
612+
var containerClient = new BlobContainerClient(ConnectionString, containerName);
613+
614+
try
615+
{
616+
containerClient.Create();
617+
BlobClient blobClient = containerClient.GetBlobClient(Randomize("sample-blob"));
618+
await blobClient.UploadAsync(BinaryData.FromString(data), new BlobUploadOptions { Metadata = initialMetadata });
619+
620+
#region Snippet:SampleSnippetsBlobMigration_EditMetadata
621+
IDictionary<string, string> metadata = blobClient.GetProperties().Value.Metadata;
622+
metadata.Add("foo", "bar");
623+
blobClient.SetMetadata(metadata);
624+
#endregion
625+
626+
var expectedMetadata = new Dictionary<string, string> { { "foo", "bar" }, { "fizz", "buzz" } };
627+
var actualMetadata = (await blobClient.GetPropertiesAsync()).Value.Metadata;
628+
Assert.AreEqual(expectedMetadata.Count, actualMetadata.Count);
629+
foreach (var expectedKvp in expectedMetadata)
630+
{
631+
Assert.IsTrue(actualMetadata.TryGetValue(expectedKvp.Key, out var actualValue));
632+
Assert.AreEqual(expectedKvp.Value, actualValue);
633+
}
634+
}
635+
finally
636+
{
637+
await containerClient.DeleteIfExistsAsync();
638+
}
639+
}
640+
641+
[Test]
642+
public async Task EditBlobWithMetadata()
643+
{
644+
string data = "hello world";
645+
var initialMetadata = new Dictionary<string, string> { { "fizz", "buzz" } };
646+
647+
string containerName = Randomize("sample-container");
648+
var containerClient = new BlobContainerClient(ConnectionString, containerName);
649+
650+
try
651+
{
652+
containerClient.Create();
653+
BlobClient blobClient = containerClient.GetBlobClient(Randomize("sample-blob"));
654+
await blobClient.UploadAsync(BinaryData.FromString(data), new BlobUploadOptions { Metadata = initialMetadata });
655+
656+
#region Snippet:SampleSnippetsBlobMigration_EditBlobWithMetadata
657+
// download blob content and metadata
658+
BlobDownloadResult blobData = blobClient.DownloadContent();
659+
660+
// modify blob content
661+
string modifiedBlobContent = blobData.Content + "FizzBuzz";
662+
663+
// reupload modified blob content while preserving metadata
664+
// not adding metadata is a metadata clear
665+
blobClient.Upload(
666+
BinaryData.FromString(modifiedBlobContent),
667+
new BlobUploadOptions() { Metadata = blobData.Details.Metadata });
668+
#endregion
669+
670+
var actualMetadata = (await blobClient.GetPropertiesAsync()).Value.Metadata;
671+
Assert.AreEqual(initialMetadata.Count, actualMetadata.Count);
672+
foreach (var expectedKvp in initialMetadata)
673+
{
674+
Assert.IsTrue(actualMetadata.TryGetValue(expectedKvp.Key, out var actualValue));
675+
Assert.AreEqual(expectedKvp.Value, actualValue);
676+
}
677+
}
678+
finally
679+
{
680+
await containerClient.DeleteIfExistsAsync();
681+
}
682+
}
683+
605684
[Test]
606685
public async Task SasBuilder()
607686
{

0 commit comments

Comments
 (0)