Skip to content

Commit 2a96313

Browse files
HarshaNalluruHarshaNalluru
authored
[Perf Tests] [Storage] Porting storage-file-share and storage-file-datalake perf tests to the latest model (Azure#14075)
### Description Makes changes to `storage-file-share` and `storage-file-datalake` based on Azure#13770 and Azure#14024 ### Old - `sdk/storage/storage-file-share/test/perfstress/track-1/` - npm project (but imports the perf framework) - `sdk/storage/storage-file-share/test/perfstress/track-2/` - depends on src code like the regular tests ### New - `sdk/storage/perf-tests/storage-file-share-track-1/` - Independent npm project (but imports the perf framework) - `sdk/storage/perf-tests/storage-file-share/`              - New project maintained through "rush" (depends on storage-file-share on master) The following walk-through for executing perf tests uses storage-file-share as an example. (Check Azure#13740 for proposal 1 - old.) ## Testing Track 1 - `rush update` - Navigate to `sdk\storage\perf-tests\storage-file-share-track-1` - Run `npm run setup` (Builds the perf package and installs it for track-1 perf tests) - Add .env file as suggested in the readme - Run the tests as suggested by readme, example `npm run perf-test:node -- StorageFileShareDownloadTest --warmup 2 --duration 7 --iterations 2 --parallel 2` ## Testing Track 2 - unpublished(current master) - `rush update` - Navigate to `sdk\storage\perf-tests\storage-file-share` - `rush build -t perf-test-storage-file-share` - Add .env file as suggested in the readme - Run the tests as suggested by readme, example `npm run perf-test:node -- StorageFileShareDownloadTest --warmup 2 --duration 7 --iterations 2 --parallel 2` ## Testing Track 2 - older versions - Example: To test 12.2.0 - Update `"@azure/storage-file-share"` version in `package.json` to `12.2.0` - Add a new exception in `common\config\rush\common-versions.json` under `allowedAlternativeVersions` - `"@azure/storage-file-share": [..., "12.2.0"]` - `rush update` (generates a new pnpm-lock file) - Navigate to `sdk\storage\perf-tests\storage-file-share` - `rush build -t perf-test-storage-file-share` - Add .env file as suggested in the readme - Run the tests as suggested by readme, example `npm run perf-test:node -- StorageFileShareDownloadTest --warmup 2 --duration 7 --iterations 2 --parallel 2`
1 parent 96fe07e commit 2a96313

39 files changed

+287
-89
lines changed

‎common/config/rush/pnpm-lock.yaml‎

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎rush.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -652,6 +652,16 @@
652652
"projectFolder": "sdk/storage/perf-tests/storage-blob",
653653
"versionPolicyName": "test"
654654
},
655+
{
656+
"packageName": "@azure-tests/perf-storage-file-share",
657+
"projectFolder": "sdk/storage/perf-tests/storage-file-share",
658+
"versionPolicyName": "test"
659+
},
660+
{
661+
"packageName": "@azure-tests/perf-storage-file-datalake",
662+
"projectFolder": "sdk/storage/perf-tests/storage-file-datalake",
663+
"versionPolicyName": "test"
664+
},
655665
{
656666
"packageName": "@azure/mixedreality-authentication",
657667
"projectFolder": "sdk/mixedreality/mixedreality-authentication",

‎sdk/storage/perf-tests/storage-blob/package.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@
1212
"@azure/storage-blob": "^12.5.0-beta.1",
1313
"@azure/test-utils-perfstress": "^1.0.0",
1414
"@types/uuid": "^8.0.0",
15-
"@types/dotenv": "8.2.0",
1615
"@types/node": "^8.0.0",
1716
"@types/node-fetch": "^2.5.0",
1817
"dotenv": "^8.2.0",

‎sdk/storage/perf-tests/storage-file-datalake/README.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
### Guide
2+
3+
1. Build the storage-blob perf tests package `rush build -t perf-storage-file-datalake`.
4+
2. Copy the `sample.env` file and name it as `.env`.
5+
3. Create a storage account and populate the `.env` file with `STORAGE_CONNECTION_STRING` variable.
6+
4. Run the tests as follows
7+
- append
8+
- `npm run perf-test:node -- StorageDFSAppendTest --warmup 2 --duration 7 --iterations 2 --parallel 2`
9+
- read
10+
- `npm run perf-test:node -- StorageDFSReadTest --warmup 2 --duration 7 --iterations 2 --parallel 2`
11+
- upload
12+
- `npm run perf-test:node -- StorageDFSUploadTest --warmup 2 --duration 7 --iterations 2 --parallel 2`
13+
- upload from file
14+
- `npm run perf-test:node -- StorageDFSUploadFromFileTest --warmup 2 --duration 7 --iterations 2 --parallel 2`

‎sdk/storage/perf-tests/storage-file-datalake/package.json‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"name": "@azure-tests/perf-storage-file-datalake",
3+
"version": "1.0.0",
4+
"description": "",
5+
"main": "",
6+
"keywords": [],
7+
"author": "",
8+
"license": "ISC",
9+
"devDependencies": {
10+
"@azure/storage-file-datalake": "^12.4.0-beta.2",
11+
"@azure/test-utils-perfstress": "^1.0.0",
12+
"@types/uuid": "^8.0.0",
13+
"@types/node": "^8.0.0",
14+
"dotenv": "^8.2.0",
15+
"uuid": "^8.3.0",
16+
"tslib": "^2.0.0",
17+
"ts-node": "^8.3.0",
18+
"typescript": "4.1.2"
19+
},
20+
"private": true,
21+
"scripts": {
22+
"perf-test:node": "ts-node test/index.spec.ts",
23+
"audit": "node ../../../common/scripts/rush-audit.js && rimraf node_modules package-lock.json && npm i --package-lock-only 2>&1 && npm audit",
24+
"build": "tsc -p .",
25+
"build:samples": "echo skipped",
26+
"build:test": "echo skipped",
27+
"check-format": "prettier --list-different --config ../../../.prettierrc.json --ignore-path ../../../.prettierignore \"src/**/*.ts\" \"test/**/*.ts\" \"*.{js,json}\"",
28+
"clean": "rimraf dist dist-esm test-dist typings *.tgz *.log",
29+
"format": "prettier --write --config ../../../.prettierrc.json --ignore-path ../../../.prettierignore \"src/**/*.ts\" \"test/**/*.ts\" \"*.{js,json}\"",
30+
"integration-test:browser": "echo skipped",
31+
"integration-test:node": "echo skipped",
32+
"integration-test": "echo skipped",
33+
"lint:fix": "eslint package.json src test --ext .ts --fix --fix-type [problem,suggestion]",
34+
"lint": "eslint package.json src test --ext .ts -f html -o storage-file-datalake-perf-test-lintReport.html || exit 0",
35+
"pack": "npm pack 2>&1",
36+
"prebuild": "npm run clean",
37+
"unit-test:browser": "echo skipped",
38+
"unit-test:node": "echo skipped",
39+
"unit-test": "echo skipped",
40+
"test:browser": "echo skipped",
41+
"test:node": "echo skipped",
42+
"test": "echo skipped"
43+
}
44+
}

‎sdk/storage/perf-tests/storage-file-datalake/sample.env‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
STORAGE_CONNECTION_STRING=<connection-string>

sdk/storage/storage-file-datalake/test/perfstress/track-2/append.spec.ts renamed to sdk/storage/perf-tests/storage-file-datalake/test/append.spec.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33

44
import { PerfStressOptionDictionary } from "@azure/test-utils-perfstress";
55
import { StorageDFSTest } from "./storageTest.spec";
6-
import { DataLakeFileClient } from "../../../src";
7-
import { generateUuid } from "@azure/core-http";
6+
import { DataLakeFileClient } from "@azure/storage-file-datalake";
7+
import { v4 as generateUuid } from "uuid";
88
interface StorageDFSAppendTestOptions {
99
size: number;
1010
}

sdk/storage/storage-file-datalake/test/perfstress/track-2/index.spec.ts renamed to sdk/storage/perf-tests/storage-file-datalake/test/index.spec.ts

File renamed without changes.

sdk/storage/storage-file-datalake/test/perfstress/track-2/read.spec.ts renamed to sdk/storage/perf-tests/storage-file-datalake/test/read.spec.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33

44
import { drainStream, PerfStressOptionDictionary } from "@azure/test-utils-perfstress";
55
import { StorageDFSTest } from "./storageTest.spec";
6-
import { DataLakeFileClient } from "../../../src";
7-
import { generateUuid } from "@azure/core-http";
6+
import { DataLakeFileClient } from "@azure/storage-file-datalake";
7+
import { v4 as generateUuid } from "uuid";
88
interface StorageDFSReadTestOptions {
99
size: number;
1010
}

sdk/storage/storage-file-datalake/test/perfstress/track-2/storageTest.spec.ts renamed to sdk/storage/perf-tests/storage-file-datalake/test/storageTest.spec.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ import {
88
StorageSharedKeyCredential,
99
DataLakeFileSystemClient,
1010
DataLakeDirectoryClient
11-
} from "../../../src";
11+
} from "@azure/storage-file-datalake";
1212

1313
// Expects the .env file at the same level as the "test" folder
1414
import * as dotenv from "dotenv";
15-
import { generateUuid } from "@azure/core-http";
15+
import { v4 as generateUuid } from "uuid";
1616
dotenv.config();
1717

1818
export abstract class StorageDFSTest<TOptions> extends PerfStressTest<TOptions> {

0 commit comments

Comments
 (0)