Optimization for gigantic amount of nodes

[MunifTanjim]

I'm trying out a Linked List based internal for NuiTree.

• feat(tree): optimize redraw for large amount of nodes MunifTanjim/nui.nvim#285

Currently tree:render() recalculates everything each time it is called. That means:

• Iterating over all the nodes to prepare the display lines
• Update the whole buffer with the newly prepared display lines

For projects with a huge number of files/folders, it becomes slow as more and more folders are expanded.

With the new implementation the goal is to:

• Be able to partially calculate display lines for the changed nodes
• Be able to update the buffer partially

I tried out neo-tree with the new NuiTree internal. But I'm facing some weird bugs.

This discussion is to figure out what's wrong...

Does neo-tree mutate any internals of NuiTree that might be causing the issue? If yes, what new methods should NuiTree expose so that neo-tree does not need to hack into NuiTree internals anymore?

P.S.: MunifTanjim/nui.nvim#285 is not usable yet.

[MunifTanjim]

cc: @cseickel (not sure who else is maintaining the plugin)

[cseickel]

@nhat-vo @miversen33 @pysan3 are the active maintainers.

[cseickel]

I'm trying out a Linked List based internal for NuiTree.
...
With the new implementation the goal is to:

• Be able to partially calculate display lines for the changed nodes
• Be able to update the buffer partially

That's awesome!

I tried out neo-tree with the new NuiTree internal. But I'm facing some weird bugs.

This discussion is to figure out what's wrong...

Does neo-tree mutate any internals of NuiTree that might be causing the issue? If yes, what new methods should NuiTree expose so that neo-tree does not need to hack into NuiTree internals anymore?

If we do, it would be within this file: https://github.com/nvim-neo-tree/neo-tree.nvim/blob/v2.x/lua/neo-tree/ui/renderer.lua

Could you describe some of the bugs? I might know where to look if I understand a little more about what breaks.

[MunifTanjim]

Phew... so I made some changes and now it's works without any changes to neo-tree.

😂 Now gotta do some benchmark to see if it actually improves performance or not.

[cseickel]

It's a showdown between lua and neovim core. I'm betting the new code wins in a large tree with small changes.

I'd guess that the majority of renders in neo-tree only change 1 character.

[miversen33]

This actually raises an interesting point.
@MunifTanjim , I haven't dug into the internals of Nui in quite some time. What are your thoughts on "per leaf" rendering as opposed to entire tree rendering?

I don't know how much effort that would be, but from purely a performance mindset, I imagine that a majority of tree's don't completely change with each render. A new branch/leaf might be added or pruned, but the overall tree will remain mostly the same.

[MunifTanjim]

What are your thoughts on "per leaf" rendering as opposed to entire tree rendering?

That is what the open PR in doing. I haven't benchmarked it yet. Once that is done, I'll merge that.

@miversen33

[miversen33]

I suspected that is what this was doing but had a tough time being completely sure :) Thanks for clarifying!