Deeply nested expressions A(B(C(D(E(…), …), …), …), …) vs. temp variables

[WillAvudim]

Unrelated

@tabatkins promised to get on with it back in August 2024 - #306
Well, it's April 2025 already, and I'm churning a lot of typescript nowadays, littered with A(B(C(D(E(param))))). Having the pipeline operator would definitely help a lot, I'm going nearly blind just from counting the parenthesis alone.

Please, guys, let's make it happen.

[gustavopch]

I propose we feed o1-pro with all the relevant information and let it decide as we humans can't even choose a topic token. I may be kidding or not.

[js-choi]

• Hey, thanks for your interest in the pipe operator. I’m still hopeful for it, any pipe operator, too. Every day that I write JavaScript (or work in a language with a pipe operator), I wish for the pipe operator to be done.
• Coincidentally, I’d been drafting an update post for the past week. I just posted it to Official changes thread #232 (comment).
• There’s been a lot of stasis, then a little recent movement in the past few months. For example, I had almost no volunteer time over the past four years to work on TC39 until recently.
• The proposal isn’t dead, but if it succeeds, it won’t be done for another year, if not two or even more.
• There’s movement in related proposals that might or might not free up # as the topic. But there may also be complications from proposed changes to the language in JS0/JSSugar. Read the update for more details.
• I’m going to close this issue, since I don’t think there’s anything actionable here. I hope that makes sense to you.
• If you read the update and have any questions, please feel free to open a new issue.

[js-choi]

I'm churning a lot of typescript nowadays, littered with A(B(C(D(E(param)))))

Actually, I’m interested in this part. One objection that some people in the Committee has given to any pipe operator is that “temporary variables might be good enough”.

• I have continued to argue that the explainer’s real-world examples (and, indeed, any real-world codebase) show clear tendencies by developers to create poorly readable deeply nested expressions, instead of reaching for temporary variables.
• Indeed, this has been the impetus for another potential usability study by TG5. See the general proposal update and Role of usability studies in the explainer #216 for more information.
• So…why don’t you just assign to temporary variables in your code “littered with A(B(C(D(E(param)))))”?
  • I think I know why—I live through it myself—but I would love to hear experiences from other people that I would be able to take to the Committee.
• Also, how often do you have n-ary calls in those deeply nested expressions?

[WillAvudim]

• I have continued to argue that the [explainer’s real-world examples][] (and, indeed, any real-world codebase) show clear tendencies by developers to create poorly readable deeply nested expressions, instead of reaching for temporary variables.
• Indeed, this has been the impetus for another potential usability study by TG5. See the general proposal update and Role of usability studies in the explainer #216 for more information.
• So…why don’t you just assign to temporary variables in your code “littered with A(B(C(D(E(param)))))”?

Today I've been rewriting sqlite queries to work against in-memory representation of data, stored in arrays and objects (perf optimization), which means I've got to (repeatedly) do filter -> apply is_null verifications and merge missing data -> group-by -> filter -> apply computation -> iterate over subgroups -> sum up the totals.
That is, a fairly simple SQL query translates into 6 (!) transitions, and the names, if you'd like to take the temporary variable route, would be disastrous. How do you imagine it? filtered_values, filtered_values_without_nulls, filtered_values_without_nulls_grouped_by_org, etc? And that's just a single query while I have to convert up to 7 per functionality module.
Thing is, temp variables names are meaningless, because essentially you'll end up just repeating the name of the operation you just did, and it's already in the code. So, now I'm forced to duplicate the op names all over the place.

• Also, how often do you have n-ary calls in those deeply nested expressions?

Oh, I have a function that, given an iterator, runs lambda functions against each of the subgroup. So, it would be a pipeline (multiple ops per subgroup) inside the pipeline over the entire dataset.

[js-choi]

Thanks for the response.

• If any of this deeply nested multi-step code is open source, or if you can share simplified versions of the code that you’re allowed to share, that would be valuable.
• We might even put it on a slide someday—or at least the explainer.
• What matters most is:
  • that it contains deeply nested expressions,
  • that it’s from the real world,
  • and that you deliberately decided not to use temporary variables in it.

This invitation extends to anyone else too: your thoughts on “just use temp variables” and any real-world code examples.

The more real-world examples to show the Committee just how big of a problem this is, the better.

[gustavopch]

I also don't like using temp variables. I'll try to explain how I think.

I like my code to be as close as possible to how the equivalent pseudocode would be. Pseudocode is usually what we intend to do and focuses on essential complexity. Actual code is what we have to do and includes accidental complexity. Temp variables are usually accidental complexity.

Code is for humans. The closest it is to natural language (without being ambiguous of course), the better. We don't name everything we talk about. We frequently use a whole sentence to describe something that doesn't have a specific name. Things that are frequently referred to naturally end up getting a name. I see it the same way in code. I don't think literally everything should be named, sometimes it's more convenient to describe it with a sentence/expression than name it. And, in those cases, naming it would probably produce a name that's basically a redundant description of the expression on the right side of the = as the examples given by @WillAvudim:

filtered_values, filtered_values_without_nulls, filtered_values_without_nulls_grouped_by_org

I see this kind of temp variable, whose names are redundant, just like this kind of comment:

// Increment counter
incrementCounter()

It's not adding any new information. We already know it's incrementing a counter because the statement says exactly that.

In some other cases, it's less about redundancy and more about "better show than tell":

const gradients = [
  from + step * 0,
  from + step * 1,
  from + step * 2,
  from + step * 3,
].map(
  from => `linear-gradient(${direction}, white ${from}%, transparent ${to}%)`,
)

I don't think naming this array would improve the readability. I'm not even sure how I'd name it... gradientFroms? I mean, IMO, not even worth wasting time trying to find a good name. The code is already readable as it is. If I had to make it even clearer, I'd just add some comments to show an example of how the values should flow, like this:

const gradients = [
  from + step * 0, // E.g.: 50 + 10 * 0
  from + step * 1, // E.g.: 50 + 10 * 1
  from + step * 2, // E.g.: 50 + 10 * 2
  from + step * 3, // E.g.: 50 + 10 * 3
].map(
  // E.g.: 50 => linear-gradient(to bottom, white 50%, transparent 100%)
  from => `linear-gradient(${direction}, white ${from}%, transparent ${to}%)`,
)

I try to write my code in a way that, just by reading the start of each statement, I can already understand what's happening. In this case, it's easy to see this statement is producing an array of gradients. If I need to understand how it's doing that, then I read the expression and see it starts with an array of 4 values and maps those values to a template literal.

I'm not saying temp variables are always bad, I'm just saying they're not useful (IMO, of course) when the name isn't meaningful and doesn't add new information that you wouldn't easily obtain by reading the expression — and sometimes comments with examples are better in improving the comprehension than names.

@js-choi BTW, thanks for the updates and explanations in the messages you recently posted here and in the other issues.

[WillAvudim]

I reviewed our project, which is about ~200 KLOC at this point, and I must admit I don't have truly great examples where the pipeline operator would be of substantial benefits. Normally, our code doesn't venture past A(B(C))).

E.g. let's take the error processing lines:

ForkAsync(dispatcher.TerminateDispatcher(new Error(`${dbg} exited with ${JSON.stringify(code_or_signal)}.`)))

It's hard to tell if rewriting them via the pipeline operator would yield more clarity:

`${dbg} exited with ${JSON.stringify(code_or_signal)}.` |> new Error(%) |> dispatcher.TerminateDispatcher |> ForkAsync

Unit test checks would definite become better:

await Match(await a.Read({ index: 0 }), 7)

because you can formulate the operation first, and pipe it into the check, thus clearly separating the op from its expected value:

await a.Read({ index: 0 }) |> await Match(%, 7)

I've made quite a bit of progress on rewriting my sqlite queries, and they are no longer at the point where they could be expressed via the pipeline operator because of micro-optimizations.

SELECT org_name, SUM(budget) AS total_budget
FROM Org
WHERE Org.last_updated < ?update_from
GROUP BY org_name

That roughly translates into:

orgs |> FilterBy(%, org => org.last_update < update_from) |> GroupBy(%, org => org.org_name) |> Map(%, group => group.values => SumBy(%, val => val.budget))

However, in practice |> would result in generation of intermediate arrays after every operation, which is inefficient.
What I truly want instead is probably LINQ, where this would be natural - https://learn.microsoft.com/en-us/dotnet/standard/linq/

var sums = 
    from org in Orgs
    where org.last_update < update_from
    group org by org.org_name into g
    select new { org_name = g.Key, total_budget = g.ToList().Sum() }

Here the data is also pipelined from one line to the next, and there is a list of predefined item transformations that mimic SQL operators. C# compiler does a lot of optimizations in order to make it robust, and nearly as fast as hand-written code.

Google has also started doing the same recently: https://cloud.google.com/blog/products/data-analytics/simplify-your-sql-with-pipe-syntax-in-bigquery-and-cloud-logging

-- Pipe Syntax 
FROM `bigquery-public-data.chicago_taxi_trips.taxi_trips`
|> EXTEND EXTRACT(YEAR FROM trip_start_timestamp) AS year
|> AGGREGATE COUNT(*) AS num_trips 
   GROUP BY year, payment_type
|> AGGREGATE AVG(num_trips) AS avg_trips_per_year GROUP BY payment_type ASC;

As you can see, the stream of items is piped through a set of distinctly predefined operations, and the query optimization engine can produce high-perf code that is hard to match if you write everything manually.
However, both LINQ and the SQL pipe syntax result in a code that doesn't necessarily flow through separate stages, and the result is compiled as a whole, without emitting intermediate arrays. I don't think we can achieve this with the current pipeline operator proposal. Maybe we are aiming too low?

[Jopie64]

However, both LINQ and the SQL pipe syntax result in a code that doesn't necessarily flow through separate stages, and the result is compiled as a whole, without emitting intermediate arrays.

At least in c#, the reason why it doesn't create intermediate arrays is because IEnumerable is evaluated lazily. In JavaScript you can achieve similar behavior using generators. And for pipelining them through operators you can use libs like IxJS. If you want you could use |> to chain the operators (although with hack pipes I'd prefer the native lib way of chaining...).

[WillAvudim]

However, both LINQ and the SQL pipe syntax result in a code that doesn't necessarily flow through separate stages, and the result is compiled as a whole, without emitting intermediate arrays.

At least in c#, the reason why it doesn't create intermediate arrays is because IEnumerable is evaluated lazily. In JavaScript you can achieve similar behavior using generators. And for pipelining them through operators you can use libs like IxJS. If you want you could use |> to chain the operators (although with hack pipes I'd prefer the native lib way of chaining...).

Linq can compile into expression trees, e.g.

Func<int, bool> comparer = num => num < 5;

is translated by the C# compiler into

Expression<Func<int, bool>> comparerExpression2 = 
  Expression.lambda<Func<int, bool>>(
    Expression.LessThan(
      numParam,
      Expression.Constant(5)),
    numParam);

And then the executor (e.g. Entity Framework, linq-to-sql) can optimize it, translate into SQL, and compile the statement. It's a whole new level of possibilities! Sources:

1. https://learn.microsoft.com/en-us/dotnet/csharp/advanced-topics/expression-trees/
2. https://youtu.be/r69ZxXgOIK4?t=552

It's unfair to compare it to straightforward passing of an object to a function. C# compiler applies custom optimizations, such as merging of Where and Select calls, preemptively stopping iteration for Any/All/First methods, etc. It's aware of the expression tree, and emits modified IL. It doesn't simply invoke IEnumerable methods lazily as one might assume. And they do a lot of smart optimizations underneath precisely because they have this internal representation as an expression tree, and the meaning of individual elements comprising it. While with the pipeline operator you're effectively piping lambdas without any possibility for a query analyzer or an external (non-javascript) optimizer to handle it.

[tabatkins]

Right, for that sort of expression analysis you need a context with bounds so the operations can be lazily transformed. Nothing about that is incompatible with pipeline (or any other function-calling syntax), you'll just be loading up the expressions into a context and then executing it at the end, rather than calling eager transformations.