Skip to content

Commit bdfcaa4

Browse files
ghkdqhrbalsghkdqhrbals
committed
update README.md
1 parent ea63a54 commit bdfcaa4

File tree

2 files changed

+36
-313
lines changed

2 files changed

+36
-313
lines changed

‎README.md‎

Lines changed: 36 additions & 313 deletions
Original file line numberDiff line numberDiff line change
@@ -1,339 +1,62 @@
1-
http benchmarker
1+
BM performance tester
22
======
3+
> Since 2024.03.07
34
4-
* Java version : [amazon corrretto 17](https://docs.aws.amazon.com/corretto/latest/corretto-17-ug/downloads-list.html)
5-
* Spring boot version : 3.2.3
5+
A enterprise level performance testing solution. Taking inspiration from [nGrinder](https://github.com/naver/ngrinder), this project aims to develop a Spring Boot application mirroring nGrinder’s functionality as closely as possible.
66

7-
This is a simple http benchmark tool that can be used to **test the performance of a server**.
7+
You can use our service in [https://www.high-load.org](https://www.high-load.org).
88

9-
* erd link : [https://www.erdcloud.com/d/MLpTGsonrqSK7ycAh](https://www.erdcloud.com/d/MLpTGsonrqSK7ycAh)
9+
BM has 2 major components.
10+
* **bm-controller**
11+
> A web application that manage test templates, user, groups, etc. Enables the performance tester to run its test.
12+
* **bm-agent**
13+
> A scalable performance tester that create the number of vuser threads, send HTTP load to target server.
1014
11-
<details>
12-
<summary>ERD</summary>
15+
And for user who confiuring own service, eureka server is used in service discovery purpose.
1316

14-
![img.png](erd.png)
1517

16-
</details>
18+
## Features
19+
* Scalable bm-agent by advertising to eureka server.
20+
* Run multiple tests concurrently and easy check agent status and their cpu usage.
21+
* Monitoring test intermediate results.
22+
* Register your test template like url, concurrent user, duration, etc. and run it. All test template is belong to group and anyone who join that group can see the result of performance test.
23+
* Provide TPS, MTTFB percentile with p99.9, p99, p95, p90, p50.
1724

25+
## What we concerned
1826

27+
* **High HTTP request durability**
1928

20-
## API design
29+
Our service should handle high load of HTTP request since our project name is performance tester. To handle multiple HTTP request, we choose webClient (which is a reactor based library). WebClient has a Netty based async non+blocking architecture, so it is a perfect solution.
2130

31+
But, **we cannot control concurrency level of webClient**. Because webClient itself provide full concurrency with no limit and no option for limit concurrency.
2232

23-
| Method | URL | Description | Role |
24-
|--------|--------------------------------|-----------------------------------------------------| --- |
25-
| POST | /api/user | Create a user | ADMIN / USER |
26-
| GET | /api/user | Get the user information | ADMIN / USER |
27-
| PUT | /api/user | Update the user information | ADMIN / USER |
28-
| POST | /api/group | Create a group | ADMIN |
29-
| GET | /api/groups | Get the list of groups | ADMIN |
30-
| GET | /api/group/{group_id} | Get the group information | ADMIN |
31-
| POST | /api/template | Create a template | ADMIN / USER |
32-
| GET | /api/templates | Get the list of template | ADMIN / USER |
33-
| GET | /api/template/{template_id} | Get the template information | ADMIN / USER |
34-
| PATCH | /api/template/{template_id} | Update a template | ADMIN / USER |
35-
| DELETE | /api/template/{template_id} | Delete a template | ADMIN / USER |
36-
| POST | /api/benchmark | Run a benchmark test | ADMIN / USER |
37-
| GET | /api/benchmark/result/{test_id} | Get the result of a benchmark test | ADMIN / USER |
38-
| GET | /api/benchmark/results | Get the list of benchmark test results within group | ADMIN / USER |
39-
| POST | /login | Login | ADMIN / USER |
40-
| POST | /logout | Logout | ADMIN / USER |
33+
Because of that reason, **we create thread as much as the number of vuser and send webClient and blocking them**. So requests are performed in parallel as many vusers and are also blocked. We used this method to control the concurrency level.
4134

35+
* **Scalable bm-agent & Real-time agent status observer**
4236

43-
* User roles
44-
* ADMIN : Can access all APIs
45-
* USER : Can access all APIs except for
46-
* `POST /user/group`
47-
* `GET /user/groups`
48-
* `GET /user/groups/{group_id}`
37+
![img.png](img/agent-status.png)
4938

50-
## API specification
51-
### `POST /api/user` [ADMIN / USER]
52-
#### Request
39+
bm-agent is an application that actually transmits HTTP to the target server.
40+
And we observe that this bm-agent consume heavy load and need to distribute the load by making it scalable across multiple worker nodes. And the bm-controller should be noticed the scaled in/out of the corresponding bm-agent.
5341

54-
```json
55-
{
56-
"id": "gyumin",
57-
"pw": "1234",
58-
"slack_webhook_url": "https://hooks.slack.com/services/...",
59-
"email": "ghkdqhrbals@gmail.com",
60-
"email_notification": true,
61-
"slack_notification": true
62-
}
63-
```
42+
So, we configured the scheduler to notify not only the currently connected bm-agent but also new bm-agent through service discovery in every second. Service discovery in the currently deployed production uses the K8S API through the `spring-cloud-kubernetes-client` library (in the docker compose distribution, service discovery is implemented through eureka).
6443

65-
### `GET /api/user`
66-
#### Response 200
67-
```json
68-
{
69-
"id": "gyumin",
70-
"slack_webhook_url": "https://hooks.slack.com/services/...",
71-
"email": "ghkdqhrbals@gmail.com",
72-
"email_notification": true,
73-
"slack_notification": true,
74-
"created_at": "2024-02-27T21:30:21.618101+09:00",
75-
"updated_at": "2024-02-27T21:30:21.618101+09:00"
76-
}
77-
```
44+
* **Concurrency**
7845

79-
#### Response 4xx
80-
```json
81-
{
82-
"error_code": "USER_NOT_FOUND",
83-
"error_message": "User not found",
84-
"error_message_detail": ""
85-
}
86-
```
46+
Again, since our application consume heavy cpu resource, concurrency is a major consideration. So we separate major logics(calculation, saving intermediate result, sse, etc.) using multiple threads and [schedulers](https://github.com/backend-tech-forge/benchmark/issues/61). We use CompletableFuture for running logics with async non-blocking.
8747

88-
### `PUT /api/user` [ADMIN / USER]
48+
* **Resource management**
8949

90-
#### Request
50+
We use multiple threads and schedulers and here, managing their resource is important(when job finished or stop, shutdown all related schedulers and threads). So we implement composite shutdown logic.
9151

92-
```json
93-
{
94-
"id": "gyumin",
95-
"group_id": "group-a", // error when write admin
96-
"slack_webhook_url": "https://hooks.slack.com/services/...",
97-
"email": "ghkdqhrbals@gmail.com",
98-
"email_notification": true,
99-
"slack_notification": true
100-
}
101-
```
52+
* **Automation**
10253

103-
### `POST /api/group` [ADMIN / USER]
54+
We also consider automation for continuous integration and deployment. see [#34](https://github.com/backend-tech-forge/benchmark/issues/34)
10455

105-
#### Request
56+
## Quick start
10657

107-
```json
108-
{
109-
"group_id": "group-a",
110-
"description": "group A test reports"
111-
}
112-
```
58+
You can configure your own service with docker compose file.
11359

114-
### `GET /api/groups` [ADMIN]
115-
116-
#### Response
117-
118-
```json
119-
120-
{
121-
"groups": [
122-
{
123-
"group_id": "group-a",
124-
"description": "group A test reports",
125-
"created_at": "2024-02-27T21:30:21.618101+09:00",
126-
"users": [ "gyumin", "user1", "user2", ... ]
127-
},
128-
{
129-
"group_id": "group-b",
130-
"description": "group B test reports",
131-
"created_at": "2024-02-27T21:30:21.618101+09:00",
132-
"users": [ "user3", "user4", "user5", ... ]
133-
}
134-
]
135-
}
136-
137-
```
138-
139-
### `GET /api/group/{group_id}` [ADMIN / USER]
140-
141-
#### Response
142-
143-
```json
144-
{
145-
"group_id": "group-a",
146-
"description": "group A test reports",
147-
"created_at": "2024-02-27T21:30:21.618101+09:00",
148-
"users": [ "gyumin", "user1", "user2", ... ]
149-
}
150-
```
151-
152-
153-
154-
155-
### `POST /api/benchmark` [ADMIN / USER]
156-
#### Request
157-
158-
```json
159-
{
160-
"url": "http://example.com:8080/api/board",
161-
"method": "POST",
162-
"headers": {
163-
"Content-Type": "application/json"
164-
},
165-
"vuser":10,
166-
"request_per_user":1000,
167-
"body": {
168-
"board_title": "...",
169-
"board_content": "...",
170-
"user_id": "..."
171-
},
172-
"prepare": {
173-
"url": "http://example.com:8080/login",
174-
"method": "POST",
175-
"headers": {
176-
"Content-Type": "application/json"
177-
},
178-
"body": {
179-
"id": "...",
180-
"password": "..."
181-
}
182-
}
183-
}
184-
```
185-
186-
#### Response 200
187-
```json
188-
{
189-
"test_id": 26,
190-
"started_at": "2024-02-27T21:30:21.618101+09:00",
191-
"finished_at": "2024-02-27T21:30:21.618101+09:00",
192-
"url": "http://example.com:8080/user",
193-
"method": "POST",
194-
"total_requests": 10000,
195-
"total_errors": 0,
196-
"total_success": 10000,
197-
"StatusCodeCount": {
198-
"200": 10000,
199-
...
200-
},
201-
"total_users": 10,
202-
"total_duration": "13s",
203-
"mttfb_average": "28.188ms",
204-
"MTTFBPercentiles": {
205-
"p50": "13.386ms",
206-
"p75": "23.908ms",
207-
"p90": "49.640ms",
208-
"p95": "103.998ms",
209-
"p99": "224.680ms"
210-
},
211-
"tps_average": 588.80,
212-
"TPSPercentiles": {
213-
"p50": 312.23,
214-
"p75": 113.19,
215-
"p90": 54.70,
216-
"p95": 23.55,
217-
"p99": 17.43
218-
}
219-
}
220-
```
221-
222-
#### Response 4xx
223-
```json
224-
{
225-
"test_id": 26,
226-
"error_code": "PREPARE_FAILED",
227-
"error_message": "Failed to prepare the test",
228-
"error_message_detail": "id or password is invalid"
229-
}
230-
```
231-
232-
#### Response 5xx
233-
```json
234-
{
235-
"test_id": 26,
236-
"error_code": "INTERNAL_SERVER_ERROR",
237-
"error_message": "",
238-
"error_message_detail": ""
239-
}
240-
```
241-
242-
### `GET /api/benchmark/result/{test_id}` [ADMIN / USER]
243-
#### Response 200
244-
```json
245-
{
246-
"test_id": 26,
247-
"started_at": "2024-02-27T21:30:21.618101+09:00",
248-
"finished_at": "2024-02-27T21:30:21.618101+09:00",
249-
"url": "http://example.com:8080/user",
250-
"method": "POST",
251-
"total_requests": 10000,
252-
"total_errors": 0,
253-
"total_success": 10000,
254-
"StatusCodeCount": {
255-
"200": 10000,
256-
...
257-
},
258-
"total_users": 10,
259-
"total_duration": "13s",
260-
"mttfb_average": "28.188ms",
261-
"MTTFBPercentiles": {
262-
"p50": "13.386ms",
263-
"p75": "23.908ms",
264-
"p90": "49.640ms",
265-
"p95": "103.998ms",
266-
"p99": "224.680ms"
267-
},
268-
"tps_average": 588.80,
269-
"TPSPercentiles": {
270-
"p50": 312.23,
271-
"p75": 113.19,
272-
"p90": 54.70,
273-
"p95": 23.55,
274-
"p99": 17.43
275-
}
276-
}
277-
```
278-
279-
#### Response 4xx
280-
```json
281-
{
282-
"test_id": 26,
283-
"error_code": "TEST_NOT_FOUND",
284-
"error_message": "Test not found",
285-
"error_message_detail": ""
286-
}
287-
```
288-
289-
#### Response 5xx
290-
```json
291-
{
292-
"test_id": 26,
293-
"error_code": "INTERNAL_SERVER_ERROR",
294-
"error_message": "",
295-
"error_message_detail": ""
296-
}
297-
```
298-
299-
### `GET /api/benchmark/results` [ADMIN / USER]
300-
301-
#### Response 200
302-
```json
303-
{
304-
"results": [
305-
{
306-
"test_id": 26,
307-
"started_at": "2024-02-27T21:30:21.618101+09:00",
308-
"finished_at": "2024-02-27T21:30:21.618101+09:00",
309-
"url": "http://example.com:8080/user",
310-
"method": "POST",
311-
"total_requests": 10000,
312-
"total_errors": 0,
313-
"total_success": 10000,
314-
"StatusCodeCount": {
315-
"200": 10000,
316-
...
317-
},
318-
"total_users": 10,
319-
"total_duration": "13s",
320-
"mttfb_average": "28.188ms",
321-
"tps_average": 588.80,
322-
},
323-
{
324-
"test_id": 25,
325-
...
326-
}
327-
]
328-
}
329-
```
330-
331-
#### Response 5xx
332-
```json
333-
{
334-
"error_code": "INTERNAL_SERVER_ERROR",
335-
"error_message": "",
336-
"error_message_detail": ""
337-
}
338-
```
60+
Run `docker compose up` in project root directory! We already make an example docker compose files for you.
33961

62+
Instead of kubernetes service discovery, we provide eureka service for someone who don't use kubernetes.

‎img/agent-status.png‎

33 KB
Loading

0 commit comments

Comments
 (0)