Skip to content

Commit 972fba7

Browse files
gfxgfx
committed
doc
1 parent 058e90d commit 972fba7

File tree

1 file changed

+99
-56
lines changed

1 file changed

+99
-56
lines changed

‎README.md‎

Lines changed: 99 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ deepStrictEqual(decode(encoded), object);
5353
- [ExtensionCodec context](#extensioncodec-context)
5454
- [Handling BigInt with ExtensionCodec](#handling-bigint-with-extensioncodec)
5555
- [The temporal module as timestamp extensions](#the-temporal-module-as-timestamp-extensions)
56+
- [Faster way to decode a large array of floating point numbers](#faster-way-to-decode-a-large-array-of-floating-point-numbers)
5657
- [Decoding a Blob](#decoding-a-blob)
5758
- [MessagePack Specification](#messagepack-specification)
5859
- [MessagePack Mapping Table](#messagepack-mapping-table)
@@ -109,17 +110,17 @@ console.log(buffer);
109110

110111
#### `EncoderOptions`
111112

112-
Name|Type|Default
113-
----|----|----
114-
extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec`
115-
context | user-defined | -
116-
useBigInt64 | boolean | false
117-
maxDepth | number | `100`
118-
initialBufferSize | number | `2048`
119-
sortKeys | boolean | false
120-
forceFloat32 | boolean | false
121-
forceIntegerToFloat | boolean | false
122-
ignoreUndefined | boolean | false
113+
| Name | Type | Default |
114+
| ------------------- | -------------- | ----------------------------- |
115+
| extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec` |
116+
| context | user-defined | - |
117+
| useBigInt64 | boolean | false |
118+
| maxDepth | number | `100` |
119+
| initialBufferSize | number | `2048` |
120+
| sortKeys | boolean | false |
121+
| forceFloat32 | boolean | false |
122+
| forceIntegerToFloat | boolean | false |
123+
| ignoreUndefined | boolean | false |
123124

124125
### `decode(buffer: ArrayLike<number> | BufferSource, options?: DecoderOptions): unknown`
125126

@@ -143,17 +144,17 @@ NodeJS `Buffer` is also acceptable because it is a subclass of `Uint8Array`.
143144

144145
#### `DecoderOptions`
145146

146-
Name|Type|Default
147-
----|----|----
148-
extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec`
149-
context | user-defined | -
150-
useBigInt64 | boolean | false
151-
rawStrings | boolean | false
152-
maxStrLength | number | `4_294_967_295` (UINT32_MAX)
153-
maxBinLength | number | `4_294_967_295` (UINT32_MAX)
154-
maxArrayLength | number | `4_294_967_295` (UINT32_MAX)
155-
maxMapLength | number | `4_294_967_295` (UINT32_MAX)
156-
maxExtLength | number | `4_294_967_295` (UINT32_MAX)
147+
| Name | Type | Default |
148+
| -------------- | -------------- | ----------------------------- |
149+
| extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec` |
150+
| context | user-defined | - |
151+
| useBigInt64 | boolean | false |
152+
| rawStrings | boolean | false |
153+
| maxStrLength | number | `4_294_967_295` (UINT32_MAX) |
154+
| maxBinLength | number | `4_294_967_295` (UINT32_MAX) |
155+
| maxArrayLength | number | `4_294_967_295` (UINT32_MAX) |
156+
| maxMapLength | number | `4_294_967_295` (UINT32_MAX) |
157+
| maxExtLength | number | `4_294_967_295` (UINT32_MAX) |
157158

158159
To skip UTF-8 decoding of strings, `rawStrings` can be set to `true`. In this case, strings are decoded into `Uint8Array`.
159160

@@ -454,6 +455,48 @@ deepStrictEqual(decoded, instant);
454455

455456
This will become default in this library with major-version increment, if the temporal module is standardized.
456457

458+
## Faster way to decode a large array of floating point numbers
459+
460+
If there are large arrays of floating point numbers in your payload, there
461+
is a way to decode it faster: define a custom extension type for `Float#Array`
462+
with alignment.
463+
464+
An extension type's `encode` method can return a function that takes a parameter
465+
`pos: number`. This parameter can be used to make alignment of the buffer,
466+
resulting decoding it much more performant.
467+
468+
See an example implementation for `Float32Array`:
469+
470+
```typescript
471+
const extensionCodec = new ExtensionCodec();
472+
473+
const EXT_TYPE_FLOAT32ARRAY = 0; // Any in 0-127
474+
extensionCodec.register({
475+
type: EXT_TYPE_FLOAT32ARRAY,
476+
encode: (object: unknown) => {
477+
if (object instanceof Float32Array) {
478+
return (pos: number) => {
479+
const bpe = Float32Array.BYTES_PER_ELEMENT;
480+
const padding = 1 + ((bpe - ((pos + 1) % bpe)) % bpe);
481+
const data = new Uint8Array(object.buffer);
482+
const result = new Uint8Array(padding + data.length);
483+
result[0] = padding;
484+
result.set(data, padding);
485+
return result;
486+
};
487+
}
488+
return null;
489+
},
490+
decode: (data: Uint8Array) => {
491+
const padding = data[0]!;
492+
const bpe = Float32Array.BYTES_PER_ELEMENT;
493+
const offset = data.byteOffset + padding;
494+
const length = data.byteLength - padding;
495+
return new Float32Array(data.buffer, offset, length / bpe);
496+
},
497+
});
498+
```
499+
457500
## Decoding a Blob
458501

459502
[`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) is a binary data container provided by browsers. To read its contents when it contains a MessagePack binary, you can use `Blob#arrayBuffer()` or `Blob#stream()`. `Blob#stream()`
@@ -495,18 +538,18 @@ The mapping of integers varies on the setting of `useBigInt64`.
495538

496539
The default, `useBigInt64: false` is:
497540

498-
Source Value|MessagePack Format|Value Decoded
499-
----|----|----
500-
null, undefined|nil|null (*1)
501-
boolean (true, false)|bool family|boolean (true, false)
502-
number (53-bit int)|int family|number
503-
number (64-bit float)|float family|number
504-
string|str family|string (*2)
505-
ArrayBufferView |bin family|Uint8Array (*3)
506-
Array|array family|Array
507-
Object|map family|Object (*4)
508-
Date|timestamp ext family|Date (*5)
509-
bigint|N/A|N/A (*6)
541+
| Source Value | MessagePack Format | Value Decoded |
542+
| --------------------- | -------------------- | --------------------- |
543+
| null, undefined | nil | null (*1) |
544+
| boolean (true, false) | bool family | boolean (true, false) |
545+
| number (53-bit int) | int family | number |
546+
| number (64-bit float) | float family | number |
547+
| string | str family | string (*2) |
548+
| ArrayBufferView | bin family | Uint8Array (*3) |
549+
| Array | array family | Array |
550+
| Object | map family | Object (*4) |
551+
| Date | timestamp ext family | Date (*5) |
552+
| bigint | N/A | N/A (*6) |
510553

511554
* *1 Both `null` and `undefined` are mapped to `nil` (`0xC0`) type, and are decoded into `null`
512555
* *2 If you'd like to skip UTF-8 decoding of strings, set `rawStrings: true`. In this case, strings are decoded into `Uint8Array`.
@@ -517,18 +560,18 @@ bigint|N/A|N/A (*6)
517560

518561
If you set `useBigInt64: true`, the following mapping is used:
519562

520-
Source Value|MessagePack Format|Value Decoded
521-
----|----|----
522-
null, undefined|nil|null
523-
boolean (true, false)|bool family|boolean (true, false)
524-
**number (32-bit int)**|int family|number
525-
**number (except for the above)**|float family|number
526-
**bigint**|int64 / uint64|bigint (*7)
527-
string|str family|string
528-
ArrayBufferView |bin family|Uint8Array
529-
Array|array family|Array
530-
Object|map family|Object
531-
Date|timestamp ext family|Date
563+
| Source Value | MessagePack Format | Value Decoded |
564+
| --------------------------------- | -------------------- | --------------------- |
565+
| null, undefined | nil | null |
566+
| boolean (true, false) | bool family | boolean (true, false) |
567+
| **number (32-bit int)** | int family | number |
568+
| **number (except for the above)** | float family | number |
569+
| **bigint** | int64 / uint64 | bigint (*7) |
570+
| string | str family | string |
571+
| ArrayBufferView | bin family | Uint8Array |
572+
| Array | array family | Array |
573+
| Object | map family | Object |
574+
| Date | timestamp ext family | Date |
532575

533576

534577
* *7 If the bigint is larger than the max value of uint64 or smaller than the min value of int64, then the behavior is undefined.
@@ -570,16 +613,16 @@ However, MessagePack can handles binary data effectively, actual performance dep
570613

571614
Benchmark on NodeJS/v22.13.1 (V8/12.4)
572615

573-
operation | op | ms | op/s
574-
----------------------------------------------------------------- | ------: | ----: | ------:
575-
buf = Buffer.from(JSON.stringify(obj)); | 1348700 | 5000 | 269740
576-
obj = JSON.parse(buf.toString("utf-8")); | 1700300 | 5000 | 340060
577-
buf = require("msgpack-lite").encode(obj); | 591300 | 5000 | 118260
578-
obj = require("msgpack-lite").decode(buf); | 539500 | 5000 | 107900
579-
buf = require("@msgpack/msgpack").encode(obj); | 1238700 | 5000 | 247740
580-
obj = require("@msgpack/msgpack").decode(buf); | 1402000 | 5000 | 280400
581-
buf = /* @msgpack/msgpack */ encoder.encode(obj); | 1379800 | 5000 | 275960
582-
obj = /* @msgpack/msgpack */ decoder.decode(buf); | 1406100 | 5000 | 281220
616+
| operation | op | ms | op/s |
617+
| ------------------------------------------------- | ------: | ---: | -----: |
618+
| buf = Buffer.from(JSON.stringify(obj)); | 1348700 | 5000 | 269740 |
619+
| obj = JSON.parse(buf.toString("utf-8")); | 1700300 | 5000 | 340060 |
620+
| buf = require("msgpack-lite").encode(obj); | 591300 | 5000 | 118260 |
621+
| obj = require("msgpack-lite").decode(buf); | 539500 | 5000 | 107900 |
622+
| buf = require("@msgpack/msgpack").encode(obj); | 1238700 | 5000 | 247740 |
623+
| obj = require("@msgpack/msgpack").decode(buf); | 1402000 | 5000 | 280400 |
624+
| buf = /* @msgpack/msgpack */ encoder.encode(obj); | 1379800 | 5000 | 275960 |
625+
| obj = /* @msgpack/msgpack */ decoder.decode(buf); | 1406100 | 5000 | 281220 |
583626

584627
Note that `JSON` cases use `Buffer` to emulate I/O where a JavaScript string must be converted into a byte array encoded in UTF-8, whereas MessagePack modules deal with byte arrays.
585628

0 commit comments

Comments
 (0)