Why are async/generators privileged here?

[tabatkins]

The big feature that makes this different from some of the other pipeline proposals is that if the function is a generator and/or async, it automagically alters how it calls the remaining chunks of the pipeline, and changes the type of the implicit function created by the pipeline to be generator and/or async as well.

This sort of functionality is useful in general, tho - there are plenty of "types" of values that I might want to have special calling behavior. For example, one function might return a Maybe value, representing a possibly-failed operation, and want to chain it to the next function in the pipeline only if the first operation succeeded...

This is a roundabout way of saying that this proposal is just giving us fmap (call, but for functors), but artificially restricted solely to the iterator and promise functors. It would be much more compelling if it figured out a way to give us arbitrary fmap and/or bind.

[TheNavigateur]

Explanation here. It's essentially centered on the way the function is semantically declared. Since these are concrete semantic constructs within JavaScript itself, it makes sense to honor their declared semantics during composition itself.

[tabatkins]

I don't understand what that link is trying to say. In particular, (1) at the link you provided needs some expansion, as I can't tell what it's trying to say. (It's particularly confusing that one of the code samples uses async while the other uses iterators - I can't tell what differences are intended to be instructive versus which are accidental.)

[TheNavigateur]

One is an AsyncFunction, and the other is a GeneratorFunction. When using either in a composition, due to the way its "output" is declared, I expect that declared output to be passed to the next function in the chain, and not the underlying Promise or Iterator. Therefore, the behaviour proposed is natural and ought to be expected in such cases. This is in contrast to an ordinary Function that returns a Promise or Iterator, which would pass that Promise/Iterator instead, precisely because the function's return value is semantically declared that way.

[tabatkins]

Oh, so they're not meant to be opposing examples, but supporting each other, ok.

This doesn't address my concern, tho - there are other types of "value containers" that would be useful to integrate into a function-calling shorthand like this, so why are iterators/asyncs specially called out here? It seems useful to have this driven by a more generic mechanism.

It also brings up a further problem - semantic intent regardless, it's generally not possible to distinguish between an iterator-returning function and a generator function, and nothing else makes this distinction. As such, libraries currently implement iterators either way, whichever seems easier for their particular case. Same, but even stronger, for async vs promise-returning - there is no particular reason to prefer this:

async function foo() { return await pFunc(); }

over this:

function foo() { return pFunc(); }

when coding - the two are identical in meaning, and the latter is shorter by twelve characters! But the former will trigger your magic upgrading, while the latter won't. That seems pretty unfortunate and magical - the user of a library needs to be intimately familiar with the internal implementation details of their library to understand what'll happen when they pipeline its functions, and the author of a library needs to be aware that there is a difference, and make an affirmative choice for each individual iterator/promise function whether it, semantically, returns an iterator/promise or multiple/async values, and code accordingly. This seems even more problematic!

Thus the thrust of my issue - this sort of functionality seems useful, but it shouldn't be magic and restricted. What we probably actually want is a small stable of pipeline variants - one does normal function calls, one does functor-map (handling iterators and promises in the magic way you want), maybe one does chain/bind; the latter two can invoke symbol-keyed methods on the object for customizability. This puts the context-switch in the author's hands, rather than keying it to an otherwise-arbitrary choice of the library authors.

[TheNavigateur]

I believe the advantages far outweigh the disadvantages.

I don't believe it's "magic" in light of how the functions are declared. It's only magic if you have to insist that they are the same as the promise/iterator returning equivalents, which they are functionally, but not semantically. If completely unsure, they also resolve to different types: AsyncFunction vs GeneratorFunction vs Function.

Can you give a usage example of your proposed approach?

[tabatkins]

There is literally no semantic difference between the two promise examples I gave - convenience wrappers around more complex function calls are common, and an otherwise-irrelevant syntax difference shouldn't have large functional differences in this one and only case.

As others have argued before, it's intentionally the case that the stdlib doesn't distinguish between AsyncFunction and a promise-returning Function, or a GeneratorFunction versus a function that returns an iterator. These are not meant to be significant differences, they're just implementation details.

[TheNavigateur]

By semantic, I mean a function's declared output value.

As you probably already know, an async function is declared to return its promised value (this probably goes without saying):

async ()=>5

This is not insignificant, and is the entire crux of why its reasonable to expect the declared output (e.g. 5 in this case) to be piped to the next function in the chain, not a promise.

The original intention of the async/generator function features are irrelevant here, as it's really all about how a typical developer in the future would logically expect the function they declared to be treated, even if they don't know anything about promises or iterators.

And that's the point: async and generator functions can be both declared and utilised without the developer having any knowledge about promises or iterators. They effectively hide those underlying implementations.

Finally, this proposal is a function composition operator. And there are only 4 types of functions in existence: Function, AsyncFunction, GeneratorFunction and AsyncGeneratorFunction. So it doesn't even necessarily make sense to extend this to other types of "things". Doing so could easily be a source of confusion and affect code's overall readability.

Of course I take your point about the library function ambiguity when not documented, but I believe that can be solved by documentation, and even in its absence is inspectable via instanceof, and I also believe promise-returning functions will eventually be a relic as functions are declared as async instead, and as I said previously I believe this factor of concern is far outweighed by the advantages.

[tabatkins]

My precise point is that the committee, very intentionally, considers it important to treat async ()=>5 and ()=>Promise.resolve(5) identical everywhere. Which you use is purely a syntax choice, not a meaningful distinction.

Any attempt to propose something that draws a significant distinction between these two cases is going to fail to pass the committee.

Finally, this proposal is a function composition operator.

Because the committee doesn't want to treat GeneratorFunction different from a function returning an iterator, or AsyncFunction different from a function returning a promise, then as written this can't be a function-composition operator; by necessity it's an operator for calling functions on at least three datatypes (functions, iterators, and promises), with different functionality for each.

[TheNavigateur]

@tabatkins I am precisely arguing that there should be a departure from that resolution, for the reasons stated. It has to be that the advantages outweigh the drawbacks, for which I think this appears to me to be a clear case. I am happy to be convinced otherwise, even though I have not yet been.

And just to be clear, the operator does not operate on promises or iterators, but on Functions, AsyncFunctions, GeneratorFunctions and AsyncGeneratorFunctions, none of which are promises or iterators.

[tabatkins]

From what I can tell, your argument for departing from that resolution is that it makes it easier to compose the functions in the correct way; you can tell before evaluating the function that it's an AsyncFunction, rather than having to wait until you see its return value is a Promise, etc.

My point in opening this thread, tho, is that:

• there's already a well-established functional pattern for chaining iterators and promises; it's the Functor and Monad patterns
• there's plenty of other functors and monads that would love to have an easy invocation/chaining method like this, most particularly the commonly-used Option and Result monads. They're used everywhere in Rust, for example.

Like, I'd love to have an (Option-returning) Map.get() method, and be able to say foo.get("key1") |> #.get("key2") (handwaving) and have it magically utilize the Option.chain method, so I either get the result of foo.get("key1").get("key2") if both successfully return, or an empty Option if either fail. Right now I have to do foo.get("key1").chain(x=>x.get("key2")), similar to the issues with chaining promises, iterators, and async iterators.

[TheNavigateur]

@tabatkins How would you accomplish what you want, as a feature in the language? I would say any implementation of support for that could be confusing to read. But you might have a way for that that is not confusing.

On the other hand, async x=>process(5) reads "`process(5) is the return value", so I would argue it's less, not more confusing, to chain the semantically declared return value, instead of a promise.

Not sure if I missed your point though...

[tabatkins]

As a direct solution, by baking in the combinator as a different glyph. If |> is "function call", then you'd need an additional glyph for "map" and one for "chain" (/"bind"), which invoke a [Functional.map] symbol on the LHS object and pass the RHS function to the result. For the sake of example, let's spell those |map> and |chain> (not serious suggestions).

Here's an example showing this as useful:

So, in today's Map, .get() returns either the requested value, or undefined. So if you have a nested map, foo.get(key1).get(key2) returns the deep value if the first map has a key1, and the associated inner map has a key2. If the inner map doesn't have a key2, you get undefined; if the outer map doesn't have a key1, you get a "Cannot read property 'get' of undefined" script error! Instead you have to either store the first result in a temp variable, test it for undefinedness, and only descend if the result is another Map, or wait for the optional-chaining proposal (with optional-call semantics) to give you the ability to write foo.get(key1).?get(key2).

If, instead, Map.get() returned an Option, which was either a Success containing a value or a Failure containing nothing, and had the monadic .chain() method on it, you would instead write foo.get(key1).chain(x=>x.get(key2)), and you'll either get a Success containing the innermost result, or a Failure if either the inner or outer map failed to have the requested key.

Translated over to monadic-pipeline, it would be written as foo.get(key1) |chain> #.get(key2), avoiding one set of wrapping parens.

Or if you just wanted to perform some operation on the result of the first .get(), assuming it succeeded, you could write it as `foo.get(key1) |map>

---

This then applies equally to iterators, async, and async iterators; the first two are monads, the second is a nested monad.

• Chain two iterators together, like 3 |> range(#) |chain> toPair(#), you get an iterator producing [0, 0, 1, 1, 2, 2]. (Or, if you wanted it to be nested, you could write 3 |> range(#) |map> toPair(#) to get [[0, 0], [1, 1], [2, 2]].)
• Chain two asyncs, like 5 |> async1(#) |chain> async2(#), you get a promise containing the result of async2(await async1(5)).
• Map an async over an iterator, like 5 |> range(#) |map> asyncDouble(#) you get an AsyncGenerator function returning an iterator of promises. (Using |chain> would result in a TypeError at runtime, as .chain() requires the result of the callback to be of the same type as the object it's being called on.)

But of course it also works with any other monad; Option, Iterator/Array, and Promise are just three of the most common monads among functional languages.