Code examples

[rybakovanton-metta]

Affected URL

https://nodejs.org/en/learn/getting-started/fetch

Describe the issue in detail:

Sorry, but examples of this type looks as code smell, like someone vomited on the page

import { Writable } from 'node:stream';

import { stream } from 'undici';

async function fetchGitHubRepos() {
  const url = 'https://api.github.com/users/nodejs/repos';

  await stream(
    url,
    {
      method: 'GET',
      headers: {
        'User-Agent': 'undici-stream-example',
        Accept: 'application/json',
      },
    },
    res => {
      let buffer = '';

      return new Writable({
        write(chunk, encoding, callback) {
          buffer += chunk.toString();
          callback();
        },
        final(callback) {
          try {
            const json = JSON.parse(buffer);
            console.log(
              'Repository Names:',
              json.map(repo => repo.name)
            );
          } catch (error) {
            console.error('Error parsing JSON:', error);
          }
          console.log('Stream processing completed.');
          console.log(`Response status: ${res.statusCode}`);
          callback();
        },
      });
    }
  );
}

fetchGitHubRepos().catch(console.error);

I know some people really like this style with all that nested inline functions and params.
But you writing example, learning pages!
it should be understandable from first sight and don't force to count brackets and commas.
And much worse, this is example from honorable node.js - later that unreadable codesmell in someones production.
yes, I made it with chatgpt, but why not use readable, understandable code, which sometime better for debugging/maintainability. Yes, with jdoc, to show developers how code can looks like (yes, here too much jdoc).
And actually chatgpt highlighted problem with buffering.

import { Writable } from 'node:stream';

/**
 * Writable stream that consumes an HTTP response body,
 * buffers it safely, and parses it as JSON once the stream ends.
 *
 * IMPORTANT:
 * - This class intentionally buffers the entire response in memory.
 * - It is suitable for "normal-sized" JSON APIs, NOT unbounded streams.
 *
 * Fixes compared to typical doc examples:
 * - Avoids string concatenation (uses Buffer chunks)
 * - Enforces a maximum response size
 * - Properly propagates stream errors
 * - Keeps response metadata (status, headers) attached
 *
 * Intended usage:
 *
 *   await stream(url, opts, res => new JsonCollector(res, { onJson }))
 */
export class JsonCollector extends Writable {
  /**
   * @param {import('undici').Dispatcher.ResponseData} res
   *   The HTTP response metadata (statusCode, headers, etc.).
   *   Provided by undici before the body stream starts.
   *
   * @param {Object} [opts]
   * @param {number} [opts.maxBytes=10485760]
   *   Maximum number of bytes allowed to be buffered.
   *   Exceeding this limit will abort the stream with an error.
   *
   * @param {boolean} [opts.expectJsonContentType=false]
   *   If true, validates that the response Content-Type contains
   *   "application/json" before parsing.
   *
   * @param {(json: any, res: any) => void} [opts.onJson]
   *   Callback invoked after successful JSON parsing.
   *   Receives parsed JSON and the original response metadata.
   *
   * @param {(res: any) => void} [opts.onDone]
   *   Optional callback invoked after successful completion.
   */
  constructor(res, opts = {}) {
    super({
      /**
       * Ensures that incoming chunks are Buffers.
       * This avoids implicit string decoding inside Node streams.
       */
      decodeStrings: true,
    });

    /** @private */
    this.res = res;

    /** @private */
    this.maxBytes = opts.maxBytes ?? 10 * 1024 * 1024; // 10 MiB default

    /** @private */
    this.expectJsonContentType = opts.expectJsonContentType ?? false;

    /** @private */
    this.onJson = opts.onJson ?? (() => {});

    /** @private */
    this.onDone = opts.onDone ?? (() => {});

    /** @private */
    this.chunks = [];

    /** @private */
    this.bytes = 0;
  }

  /**
   * Called by the stream machinery for each incoming data chunk.
   *
   * Responsibilities:
   * - Track total size
   * - Enforce maxBytes limit
   * - Store raw Buffers (no string concatenation)
   *
   * @param {Buffer} chunk
   * @param {BufferEncoding} _enc
   * @param {(err?: Error) => void} cb
   * @private
   */
  _write(chunk, _enc, cb) {
    try {
      const buf = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);

      this.bytes += buf.length;
      if (this.bytes > this.maxBytes) {
        const err = new Error(
          `Response too large: ${this.bytes} bytes (limit ${this.maxBytes})`
        );
        this.destroy(err);
        return cb(err);
      }

      this.chunks.push(buf);
      cb();
    } catch (err) {
      this.destroy(err);
      cb(err);
    }
  }

  /**
   * Called exactly once when the upstream stream ends.
   *
   * Responsibilities:
   * - Validate Content-Type (optional)
   * - Concatenate buffered data
   * - Decode UTF-8
   * - Parse JSON
   * - Invoke user callbacks
   *
   * @param {(err?: Error) => void} cb
   * @private
   */
  _final(cb) {
    try {
      if (this.expectJsonContentType) {
        const ct = this._getHeader('content-type') || '';
        if (!ct.toLowerCase().includes('application/json')) {
          throw new Error(`Unexpected content-type: ${ct || '(missing)'}`);
        }
      }

      const text = Buffer.concat(this.chunks).toString('utf8');
      const json = JSON.parse(text);

      this.onJson(json, this.res);
      this.onDone(this.res);

      cb();
    } catch (err) {
      this.destroy(err);
      cb(err);
    }
  }

  /**
   * Called when the stream is destroyed or errors.
   * Used to release buffered memory eagerly.
   *
   * @param {Error|null} err
   * @param {(err?: Error|null) => void} cb
   * @private
   */
  _destroy(err, cb) {
    this.chunks = [];
    this.bytes = 0;
    cb(err);
  }

  /**
   * Retrieve a response header in a case-insensitive way.
   * Supports both plain-object headers and Headers-like interfaces.
   *
   * @param {string} name
   * @returns {string|undefined}
   * @private
   */
  _getHeader(name) {
    const h = this.res?.headers;
    if (!h) return undefined;

    if (typeof h.get === 'function') return h.get(name);

    const key = Object.keys(h).find(
      (k) => k.toLowerCase() === name.toLowerCase()
    );
    return key ? h[key] : undefined;
  }
}

and usage

import { stream } from 'undici';
import { JsonCollector } from './JsonCollector.js';

const url = 'https://api.github.com/users/nodejs/repos';

const req = {
  method: 'GET',
  headers: {
    'User-Agent': 'undici-stream-example',
    Accept: 'application/json',
  },
};

await stream(url, req, (res) => new JsonCollector(res, {
  maxBytes: 2 * 1024 * 1024,             // 2 MiB cap
  expectJsonContentType: true,           // optional
  onJson: (json, r) => {
    console.log('Repository Names:', json.map((repo) => repo.name));
    console.log('status:', r.statusCode);
  },
}));

it produced as two separate files, but it can be combined in one and import removed, to make it a little simpler.

[avivkeller]

Sorry, but examples of this type looks as code smell, like someone vomited on the page
But you writing example, learning pages!
it should be understandable from first sight and don't force to count brackets and commas.
And much worse, this is example from honorable node.js - later that unreadable codesmell in someone's production.

I'm not a huge fan of your tone here. All the contributors here put a lot of effort into this site, and it's quite of you to ask us to make a change after calling our previous iteration "vomit".

This is an open source repository, meaning anyone is free to contribute. That also means, if you'd like to see this change, you should open a PR with your suggestion.

[avivkeller]

Also, for what it's worth, the ChatGPT example you provided appears to be multitudes more complex than our current example, which, as far as I can tell, appears fairly straightforward.

If you can find a definable issue with the current example (not just that you don't like it, or ChatGPT spit out something "better"), feel free to, like I said, open a PR improving it.

[rybakovanton-metta]

It’s sad that just one word found resonance. It was not meant as an insult. I know this is open source, and contributors do a lot of hard work — not only on the main product, but also on documentation and examples, because they need to be designed, written, run, and tested.
The hard work of the community helped Node.js become famous and respected, and it sets the tone for developers around the world. People look up to Node.js code and examples.
The first message was not meant to start a holy war about code style. I’m sure the contributors of that page are experts and know what they are doing. I’m also sure the example works well, and that this style was possibly chosen to minimize the surface of the example.
At the same time, not everyone has the mindset to work with this kind of code. Many people copy/paste examples into their own products (how can this be “bad”? — this is a Node.js example!) and then don’t understand how to debug it, how to maintain it, or how to maintain it over time.
As I said, developers look up to Node.js and try to do the same things, even if their expertise is not at the same level. This is why — because Node.js is big, famous, and respected — examples, at least on learning pages, should be more friendly and show not only minimal working examples, but also build habits that support good coding style, readability, and self-documenting code.
To my mind, contributors and the community in general do not see the problem from this angle. I’m pretty sure that if they did, someone would have written a guideline.
PS: Maybe the community does not care about fair-to-middling developers. To use Node.js, you need to be good enough — and that position is also fine.
PPS: I didn’t try to show how good ChatGPT can write code instead of me. It was just an example of how it could be. I am a lazy person, and that’s why I only criticize.
PPPS: closer to original code

import { Writable } from 'node:stream';

class JsonCollector extends Writable {
  constructor(res) {
    super();
    this.res = res;
    this.buffer = '';
  }

  _write(chunk, _enc, cb) {
    this.buffer += chunk.toString('utf8');
    cb();
  }

  _final(cb) {
    try {
      const json = JSON.parse(this.buffer);
      console.log('Repository Names:', json.map(r => r.name));
    } catch (e) {
      console.error(e);
    }
    console.log('status:', this.res.statusCode);
    cb();
  }
}

usage

await stream(url, req, res => new JsonCollector(res));

[avivkeller]

At the same time, not everyone has the mindset to work with this kind of code. Many people copy/paste examples into their own products (how can this be “bad”? — this is a Node.js example!) and then don’t understand how to debug it, how to maintain it, or how to maintain it over time.

examples, at least on learning pages, should be more friendly and show not only minimal working examples, but also build habits that support good coding style, readability, and self-documenting code.

Can you explain why the current example is not "friendly" and "minimal"? To me, it seems like it clearly does what it's described to do.

Whereas, the example you provided adds a whole new Class, for instance

[efekrskl]

I agree that the docs in the "Getting Started" section could be more beginner-friendly, I've raised a similar point before and there's some WIP stuff going on, but I must disagree with everything else you claim. I know Aviv stated this already, but your tone overall (especially with that particular opening line) is more like "blaming" than looking for an improvement. I think whatever LLM tool you are using does more harm than good for your arguments, like:

The first message was not meant to start a holy war about code style

Maybe the community does not care about fair-to-middling developers. To use Node.js, you need to be good enough — and that position is also fine.

These are some bizarre claims, really. Noone is attacking you, but you are kinda attacking what's there.

Your code examples are just using a different paradigm. That doesn't necessarily make them more beginner-friendly. In fact, I'd argue that it has the opposite effect.