Add docs on running live tests serially (Azure#33829)

* General clean-up of markdownlint issues

* Add section on serial live tests

Author: heaths

sdk/core/Azure.Core.TestFramework/README.md

@@ -1,4 +1,5 @@
 # Using the TestFramework
+
 To start using the Test Framework, add a project reference using the alias `AzureCoreTestFramework` into your test `.csproj`:
 
 ``` xml
@@ -9,8 +10,8 @@ To start using the Test Framework, add a project reference using the alias `Azur
 ...
 
 </Project>
-
 ```
+
 As an example, see the [Template](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/template/Azure.Template/tests/Azure.Template.Tests.csproj#L15) project.
 
 ## Sync-async tests
@@ -30,7 +31,6 @@ public class ConfigurationLiveTests: ClientTestBase
                 ...,
                 InstrumentClientOptions(
                     new ConfigurationClientClientOptions())));
-    }
 
     public async Task DeleteSettingNotFound()
     {
@@ -50,7 +50,6 @@ When using sync-async tests with recorded tests two sessions files will be gener
 
 You can disable the sync-forwarding for an individual test by applying the `[AsyncOnly]` attribute to the test method.
 
-
 __Limitation__: all method calls/properties that are being used have to be `virtual`.
 
 ## Test environment and live test resources
@@ -70,7 +69,7 @@ public class AppConfigurationTestEnvironment : TestEnvironment
 }
 ```
 
-**NOTE:** Make sure that variables containing secret values are not recorded or are sanitized.
+__NOTE:__ Make sure that variables containing secret values are not recorded or are sanitized.
 
 To sanitize variables use the `options` parameter of `GetRecordedVariable`:
 
@@ -147,31 +146,53 @@ public class AppConfigurationTestEnvironment : TestEnvironment
 }
 ```
 
+### Running live tests serially
+
+By default, NUnit does not run tests within each assembly in parallel, but this be [configured](https://docs.nunit.org/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.html).
+Especially for unit tests, this is often desirable; however, live and [recorded tests](#recorded-tests) may run into some issues. Thus, by default, the `RecordedTestBase` described below is attributed
+as `[NonParallelizable]`.
+
+However, when projects are built and tested in CIs, all projects are testing in parallel. This means, for example, you can have two or more assemblies running tests such as one backing up or restoring
+a resource while another assembly's tests are trying to use that resource. The service may return an error like HTTP 409.
+
+To isolate one or more projects so that they are tested serially, add a _service.projects_ file to your service directory e.g., _sdk/keyvault/service.projects_ with content like the following to set the
+`TestInParallel` metadata to `false`:
+
+```xml
+<Project>
+  <ItemGroup>
+    <ProjectReference Update="$(MSBuildThisFileDirectory)Azure.Security.KeyVault.Administration/tests/*.csproj">
+        <TestInParallel>false</TestInParallel>
+    </ProjectReference>
+  </ItemGroup>
+</Project>
+```
+
 ## Test settings
 
 Test settings can be configured via `.runsettings` files. See [nunit.runsettings](https://github.com/Azure/azure-sdk-for-net/blob/main/eng/nunit.runsettings) for available knobs.
 
 There are two ways to work with `.runsettings`. Both are picked up by Visual Studio without restart.
+
 - You can edit [nunit.runsettings](https://github.com/Azure/azure-sdk-for-net/blob/main/eng/nunit.runsettings) locally to achieve desired configuration.
 - You can prepare few copies of `.runsettings` by cloning [nunit.runsettings](https://github.com/Azure/azure-sdk-for-net/blob/main/eng/nunit.runsettings).
+
 Load them in Visual Studio (`Test>Configure Run Settings` menu) and switch between them. This option requires setting an environment variable `AZURE_SKIP_DEFAULT_RUN_SETTINGS=true`.
 
 ## TokenCredential
 
 If a test or sample uses `TokenCredential` to construct the client use `TestEnvironment.Credential` to retrieve it.
 
 ``` C#
-    public abstract class KeysTestBase : RecordedTestBase<KeyVaultTestEnvironment>
-    {
-        internal KeyClient GetClient() =>
-            InstrumentClient(
-                new KeyClient(
-                    new Uri(TestEnvironment.KeyVaultUrl),TestEnvironment.Credential,
-                    InstrumentClientOptions(
-                        new KeyClientOptions())));
-        }
-    }
-
+public abstract class KeysTestBase : RecordedTestBase<KeyVaultTestEnvironment>
+{
+    internal KeyClient GetClient() =>
+        InstrumentClient(
+            new KeyClient(
+                new Uri(TestEnvironment.KeyVaultUrl),TestEnvironment.Credential,
+                InstrumentClientOptions(
+                    new KeyClientOptions())));
+}
 ```
 
 ## Recorded tests
@@ -180,7 +201,6 @@ The test framework provides an ability to record HTTP requests and responses and
 
 To use recorded test functionality inherit from `RecordedTestBase<T>` class and use `InstrumentClientOptions` method when creating the client instance. Pass the test environment class as a generic argument to `RecordedTestBase`.
 
-
 ``` C#
 public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnvironment>
 {
@@ -215,7 +235,7 @@ In development scenarios where it's required to change mode quickly without rest
 Recorded tests can be attributed with the `RecordedTestAttribute` in lieu of the standard `TestAttribute` to enable functionality to automatically re-record tests that fail due to recording session file mismatches.
 Tests that are auto-rerecorded will fail with the following error and succeed if re-run.
 
-```
+```text
 Error Message:
    Test failed playback, but was successfully re-recorded (it should pass if re-run). Please copy updated recording to SessionFiles.
 ```
@@ -233,6 +253,7 @@ public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnviro
     }
 }
 ```
+
 In addition to the auto-rerecording functionality, using the RecordedTestAttribute also will automatically retry tests that fail due due to exceeding the global test time limit.
 
 When running tests in `Live` or `Record` mode, the Test Framework will prompt you to create the live test resources required for the tests if you don't have environment variables or an env file containing the required variables needed for the tests. This means that you do not have to manually run the New-TestResources script when attempting to run live tests! The Test Framework will also attempt to automatically extend the expiration of the test resource resource group whenever live tests are run. If the resource group specified in your .env file or environment variable has already expired and thus been deleted, the framework will prompt you to create a new resource group just like it would if an env variable required by the test was missing.
@@ -244,19 +265,19 @@ When tests are run in recording mode, session records are saved to the project d
 ### Sanitizing
 
 Secrets that are part of requests, responses, headers, or connections strings should be sanitized before saving the record.
-**Do not check in session records containing secrets.** Common headers like `Authentication` are sanitized automatically, but if custom logic is required and/or if request or response body need to be sanitized, the `Sanitizer` property should be used as an extension point.
+__Do not check in session records containing secrets.__ Common headers like `Authentication` are sanitized automatically, but if custom logic is required and/or if request or response body need to be sanitized, the `Sanitizer` property should be used as an extension point.
 
 For example:
 
 ```C#
-    public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnvironment>
+public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnvironment>
+{
+    public ConfigurationLiveTests()
     {
-        public ConfigurationLiveTests()
-        {
-            SanitizedHeaders.Add("example-header");
-            SanitizeQueryParameters.Add("example-query-parameter");
-        }
+        SanitizedHeaders.Add("example-header");
+        SanitizeQueryParameters.Add("example-query-parameter");
     }
+}
 ```
 
 Another sanitization feature that is available involves sanitizing Json payloads.
@@ -265,21 +286,20 @@ By adding a [Json Path](https://www.newtonsoft.com/json/help/html/QueryJsonSelec
 By default, the following values are added to the `JsonPathSanitizers` to be sanitized: `primaryKey`, `secondaryKey`, `primaryConnectionString`, `secondaryConnectionString`, and `connectionString`.
 
 ```c#
-    public class FormRecognizerLiveTests: RecordedTestBase<FormRecognizerTestEnvironment>
+public class FormRecognizerLiveTests: RecordedTestBase<FormRecognizerTestEnvironment>
+{
+    public FormRecognizerLiveTests()
     {
-        public FormRecognizerLiveTests()
-        {
-            JsonPathSanitizers.Add("$..accessToken");
-            JsonPathSanitizers.Add("$..source");
-        }
+        JsonPathSanitizers.Add("$..accessToken");
+        JsonPathSanitizers.Add("$..source");
     }
+}
 ```
 
 ### Matching
 
 When tests are run in `Playback` mode, the HTTP method, Uri, and headers are used to match the request to the recordings. Some headers change on every request and are not controlled by the client code and should be ignored during matching. Common headers like `Date`, `x-ms-date`, `x-ms-client-request-id`, `User-Agent`, `Request-Id` are ignored by default but if more headers need to be ignored, use the various matching properties to customize as needed.
 
-
 ``` C#
     public class ConfigurationLiveTests: RecordedTestBase<AppConfigurationTestEnvironment>
     {
@@ -373,7 +393,7 @@ How it looks it the test explorer:
 
 ![image](https://user-images.githubusercontent.com/1697911/72942831-52c7ca00-3d29-11ea-9b7e-2e54198d800d.png)
 
-**Note:** If test recordings are enabled, the recordings will be generated against the latests version of the service.
+__Note:__ If test recordings are enabled, the recordings will be generated against the latests version of the service.
 
 ## Support for an additional test parameter
 
@@ -395,6 +415,7 @@ public class TableServiceLiveTestsBase : RecordedTestBase<TablesTestEnvironment>
     {
         _endpointType = endpointType;
     }
+}
 ```
 
 ```c#
@@ -413,9 +434,10 @@ public class TableServiceLiveTestsBase : RecordedTestBase<TablesTestEnvironment>
         _serviceVersion = serviceVersion;
         _endpointType = endpointType;
     }
+}
 ```
 
-**Note:** Additional parameter options work with test recordings and will create differentiated SessionRecords test class directory names for each additional parameter option.
+__Note:__ Additional parameter options work with test recordings and will create differentiated SessionRecords test class directory names for each additional parameter option.
 For example:
 
 `/SessionRecords/TableClientLiveTests(CosmosTable)/CreatedCustomEntitiesCanBeQueriedWithFiltersAsync.json`
@@ -449,7 +471,7 @@ To download and unpack all artifacts use the `Download-DevOpsRecordings.ps1` scr
 
 The `Download-DevOpsRecordings.ps1` would wait for active runs to finish before retrieving artifacts unless `-NoWait` switch is used.
 
-**NOTE:** these scripts require being signed in with Azure CLI (https://docs.microsoft.com/cli/azure/authenticate-azure-cli?view=azure-cli-latest) and access to the internal DevOps project (https://dev.azure.com/azure-sdk/internal/)
+__NOTE:__ these scripts require being [signed in with Azure CLI](https://docs.microsoft.com/cli/azure/authenticate-azure-cli?view=azure-cli-latest) and access to the [internal DevOps project](https://dev.azure.com/azure-sdk/internal/).
 
 ### Note on private/non-virtual fields in your clients (such as _clientDiagnostics) and InternalsVisibleTo
 
@@ -490,8 +512,6 @@ There are various helpful classes that assist in writing tests for the Azure SDK
 [TestEnvVar](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/core/Azure.Core.TestFramework/src/TestEnvVar.cs) allows you to wrap a block of code with a using statement inside which the configured Environment variables will be set to your supplied values.
 It ensures that the existing value of any configured environment variables are preserved before they are set them and restores them outside the scope of the using block.
 
-#### Example usage
-
 ```c#
 using (var _ = new TestEnvVar("AZURE_TENANT_ID", "foo"))
 {
@@ -507,8 +527,6 @@ using (var _ = new TestEnvVar("AZURE_TENANT_ID", "foo"))
 It ensures that the existing value of any configured switches are preserved before they are set them and restores them outside the scope of the using block.
 Note: Even if an `AppContext` switch was un-set prior to setting it via `TestAppContextSwitch`, it will be unset after leaving the scope of the using block.
 
-#### Example usage
-
 ```c#
 var isSet = AppContext.TryGetSwitch("Azure.Core.Pipeline.DisableHttpWebRequestTransport", out val))
 // isSet is false
@@ -526,9 +544,9 @@ var isSet = AppContext.TryGetSwitch("Azure.Core.Pipeline.DisableHttpWebRequestTr
 ```
 
 ### AsyncAssert
+
 This type contains static helper methods that cover some of the gaps in NUnit when it comes to async assertions. For instance, attempting to assert that a specific exception is thrown using Assert.That, Assert.Throws, or Assert.ThrowsAsync all result in sync over async code, which can lead to test flakiness.
 
-#### Example usage
 ```c#
 ServiceBusException exception = await AsyncAssert.ThrowsAsync<ServiceBusException>(
     async () => await args.CompleteMessageAsync(message, args.CancellationToken));

sdk/synapse/Azure.ResourceManager.Synapse/tests/ScenarioTests/WorkspaceOperationTests.cs

@@ -26,6 +26,7 @@ public async Task Initialize()
 
         [Test]
         [RecordedTest]
+        [Ignore("https://github.com/Azure/azure-sdk-for-net/issues/33867")]
         public async Task TestWorkspaceLifeCycle()
         {
             // create workspace