Commit ab9387c
authored
feat: reduce memory allocation for common slice and map types (#30)
This commit reduces memory allocations for some collection types by avoiding the slow
and expensive JSON marshal->unmarshal path. It extends the list of type
assertions to slices and maps of some primitive types.
**Additional context**
While memory profiling a production application we (@dyolcekaj) noticed that 50+% of
our memory allocations were happening during
`ldvalue.CopyArbitraryValue`. We use feature flags heavily, and for any
HTTP request it is typical to evaluate 2-8 feature flags. We found that
the reason for this high memory usage was due to a string slice with
several hundred items being passed through `ldvalue.CopyArbitraryValue`
on most `ldcontext.Builder` instantiations. This was occurring in a
general use function like
```
func IsEnabled(ctx context.Context, flag string, attributes map[string]any) bool {
client := ctx.Value("ld-client).(*ldclient.LDClient)
builder := ldcontextBuilderFromCtx(ctx) // get some HTTP request type info, user info, etc.
// note ldvalue.CopyArbitraryValueMap has the same issue
for key, value := range attributes {
builder.SetValue(key, ldvalue.CopyArbitraryValue(value))
}
// ignore errors for ex.
enabled, _ := client.BoolVariation(flag, builder.Build(), false)
return enabled
}
```
If any of the `map[string]any` values are themselves a `[]string` or
similar we hit the `FromJSONMarshal` slow path. To partially solve the
problem we have written a wrapper around this SDK with more or less the
same code as I am submitting here. The reduction in memory allocation is
especially noticeable when copying `[]string`, but in all cases there is
a fairly significant improvement in latency and total memory usage.
Although this is verbose, alternatives that use generic functions like
```
func copyArbitraryType[T comparable](data []T) Value { ... }
```
don't save on allocations, and using reflection to determine the slice
or map element types does have some benefit especially for slices but
still has a high number of allocations for large maps. You can see an
implementation of this change using reflection
[here](https://github.com/dyolcekaj/launchdarkly-go-sdk-common/tree/arbitrary-collection-copies)
Here are benchmarks from my machine showing the improvement for these
specific use cases. Command for all is
`go test -benchmem '-run=^$$' -bench="CollectionCopy*" ./ldvalue`
Before:
```
BenchmarkCollectionCopyMapStringSmall-16 197137 5597 ns/op 4730 B/op 47 allocs/op
BenchmarkCollectionCopyMapStringLarge-16 1776 632766 ns/op 676664 B/op 4043 allocs/op
BenchmarkCollectionCopySliceStringSmall-16 415045 2731 ns/op 3931 B/op 19 allocs/op
BenchmarkCollectionCopySliceStringMedium-16 57158 20780 ns/op 35357 B/op 112 allocs/op
BenchmarkCollectionCopySliceStringLarge-16 6002 186840 ns/op 267965 B/op 1016 allocs/op
BenchmarkCollectionCopySliceIntSmall-16 428552 2633 ns/op 3835 B/op 9 allocs/op
BenchmarkCollectionCopySliceIntMedium-16 58148 20332 ns/op 34397 B/op 12 allocs/op
BenchmarkCollectionCopySliceIntLarge-16 6182 180290 ns/op 260010 B/op 15 allocs/op
```
After:
```
BenchmarkCollectionCopyMapStringSmall-16 1568584 769.4 ns/op 2117 B/op 2 allocs/op
BenchmarkCollectionCopyMapStringLarge-16 15642 76328 ns/op 254033 B/op 3 allocs/op
BenchmarkCollectionCopySliceStringSmall-16 5143736 226.0 ns/op 1048 B/op 2 allocs/op
BenchmarkCollectionCopySliceStringMedium-16 825129 1439 ns/op 9752 B/op 2 allocs/op
BenchmarkCollectionCopySliceStringLarge-16 78724 15250 ns/op 98328 B/op 2 allocs/op
BenchmarkCollectionCopySliceIntSmall-16 5327779 224.1 ns/op 1048 B/op 2 allocs/op
BenchmarkCollectionCopySliceIntMedium-16 832033 1419 ns/op 9752 B/op 2 allocs/op
BenchmarkCollectionCopySliceIntLarge-16 86389 13898 ns/op 98328 B/op 2 allocs/op
```
Reflection based benchmark
```
BenchmarkReflectCopyMapStringSmall-16 516852 2229 ns/op 2678 B/op 23 allocs/op
BenchmarkReflectCopyMapStringLarge-16 5341 197526 ns/op 310622 B/op 2004 allocs/op
BenchmarkReflectCopySliceStringSmall-16 2635124 447.3 ns/op 1048 B/op 2 allocs/op
BenchmarkReflectCopySliceStringMedium-16 346424 3233 ns/op 9752 B/op 2 allocs/op
BenchmarkReflectCopySliceStringLarge-16 38649 31042 ns/op 98329 B/op 2 allocs/op
BenchmarkReflectCopySliceIntSmall-16 2634298 448.7 ns/op 1048 B/op 2 allocs/op
BenchmarkReflectCopySliceIntMedium-16 340862 3293 ns/op 9752 B/op 2 allocs/op
BenchmarkReflectCopySliceIntLarge-16 39921 30468 ns/op 98328 B/op 2 allocs/op
```
Benchstat
```
goos: darwin
goarch: amd64
pkg: github.com/launchdarkly/go-sdk-common/v3/ldvalue
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
│ original.txt │ updated.txt │
│ sec/op │ sec/op vs base │
CollectionCopyMapStringSmall-16 5.534µ ± ∞ ¹ 2.103µ ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopyMapStringLarge-16 680.4µ ± ∞ ¹ 193.5µ ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringSmall-16 2818.0n ± ∞ ¹ 431.1n ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringMedium-16 20.874µ ± ∞ ¹ 3.299µ ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringLarge-16 188.00µ ± ∞ ¹ 30.97µ ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntSmall-16 2672.0n ± ∞ ¹ 441.5n ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntMedium-16 20.542µ ± ∞ ¹ 3.196µ ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntLarge-16 181.99µ ± ∞ ¹ 30.73µ ± ∞ ¹ ~ (p=1.000 n=1) ²
geomean 28.34µ 5.449µ -80.77%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
│ original.txt │ updated.txt │
│ B/op │ B/op vs base │
CollectionCopyMapStringSmall-16 4.618Ki ± ∞ ¹ 2.615Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopyMapStringLarge-16 661.2Ki ± ∞ ¹ 303.3Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringSmall-16 3.839Ki ± ∞ ¹ 1.023Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringMedium-16 34.526Ki ± ∞ ¹ 9.523Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringLarge-16 262.07Ki ± ∞ ¹ 96.02Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntSmall-16 3.745Ki ± ∞ ¹ 1.023Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntMedium-16 33.590Ki ± ∞ ¹ 9.523Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntLarge-16 253.86Ki ± ∞ ¹ 96.02Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
geomean 36.83Ki 12.74Ki -65.41%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
│ original.txt │ updated.txt │
│ allocs/op │ allocs/op vs base │
CollectionCopyMapStringSmall-16 47.00 ± ∞ ¹ 23.00 ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopyMapStringLarge-16 4.043k ± ∞ ¹ 2.004k ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringSmall-16 19.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringMedium-16 112.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceStringLarge-16 1016.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntSmall-16 9.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntMedium-16 12.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
CollectionCopySliceIntLarge-16 15.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
geomean 71.27 6.438 -90.97%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
```1 parent 6279f2c commit ab9387c
File tree
6 files changed
+581
-0
lines changed- ldvalue
6 files changed
+581
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
248 | 248 | | |
249 | 249 | | |
250 | 250 | | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
255 | 255 | | |
256 | 256 | | |
257 | 257 | | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
258 | 339 | | |
259 | 340 | | |
260 | 341 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
296 | 296 | | |
297 | 297 | | |
298 | 298 | | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
299 | 327 | | |
300 | 328 | | |
301 | 329 | | |
| |||
310 | 338 | | |
311 | 339 | | |
312 | 340 | | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
313 | 369 | | |
314 | 370 | | |
315 | 371 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
0 commit comments