Actor2d/overlay layering support

[sankhesh]

Context

vtkActor2D is a prop that can be used to render overlay/underlay geometry with the existing 3D view. When multiple 2D actors are added to the renderer, the order of rendering was dependent on the order in which they were added. However, this can be problematic in declarative style applications like react-vtk-js.

This change adds support for the layerNumber property that can be used to set the order in which the props are overlaid. 2D actors with higher layer numbers are rendered on top (a.k.a in the front).

Results

The following code adds two 2D actors.

renderer.addActor2D(sphere2D);
renderer.addActor2D(cone2D);

By default, both the props have a layerNumber == 0, and the cone will be rendered on top.

However, when the layerNumber is set,

sphere2D.setLayerNumber(3);
cone2D.setLayerNumber(2);

the cone is pushed back.

Moreover, the foreground 2D actors are sorted independently of the background 2D actors.

  const actor2D = gc.registerResource(vtkActor2D.newInstance());
  actor2D.getProperty().setColor([0.5, 0.8, 0]);
  actor2D.getProperty().setOpacity(0.3);
  actor2D.getProperty().setDisplayLocation(DisplayLocation.FOREGROUND);
  actor2D.setLayerNumber(2);
  renderer.addActor2D(actor2D);

  const actor2D1 = gc.registerResource(vtkActor2D.newInstance());
  actor2D1.getProperty().setColor([0.1, 0.8, 0.5]);
  actor2D1.getProperty().setDisplayLocation(DisplayLocation.BACKGROUND);
  renderer.addActor2D(actor2D1);
  actor2D1.setLayerNumber(1);

  const actor2D2 = gc.registerResource(vtkActor2D.newInstance());
  actor2D2.getProperty().setColor([0.8, 0.4, 0.4]);
  actor2D2.getProperty().setOpacity(1.0);
  actor2D2.getProperty().setDisplayLocation(DisplayLocation.FOREGROUND);
  actor2D2.setLayerNumber(1);
  renderer.addActor2D(actor2D2);

In this case, there are two spheres in the foreground and one in the background. The sphere added second to the renderer has a lower layer number in the foreground, hence it gets rendered below the other foreground sphere but above the background sphere.

Changes

• Documentation and TypeScript definitions were updated to match those changes

PR and Code Checklist

• semantic-release commit messages
• Run npm run reformat to have correctly formatted code

Testing

• This change adds or fixes unit tests
• Tested environment:
  • vtk.js:
  • OS:
  • Browser:

[finetjul]

Can you make a test ?

I doubt it will work when you change the layer number after a first rendering.

getViewPropsWithNestedProps() is consumed by addMissingNodes() which completes missing nodes but does not reorder children.
Does it ?

[sankhesh]

Good point @finetjul. I'll fix that. Meanwhile, do you know what's up with the CI?

[finetjul]

Good point @finetjul. I'll fix that. Meanwhile, do you know what's up with the CI?

dunno. @floryst ?

[finetjul]

I feel that deleting a node to recreate it might lead to problems in the future.

I wonder if addMissingChildren() or addMissingNodes() could be enhanced (with a new parameter? ) to optionally "sort" the children to match the provided list.

I feel that this ordering could be also extended to apply to any prop/actor...

[sankhesh]

I wonder if addMissingChildren() or addMissingNodes() could be enhanced (with a new parameter? ) to optionally "sort" the children to match the provided list.

This is a special case that requires sorting of children (vtkProp and that too the specialized vtkActor2D).

I feel that this ordering could be also extended to apply to any prop/actor..

It could be but I don't see the benefit. All 3D props use the z-buffer for sorting. This is just a special case where we have 2D over/underlays that need to be sorted.

I guess, you could argue a case where you want to mix-in 2D props with 3D ones but that can be done with the existing vtkActor and z-buffer mechanism.

[finetjul]

publicAPI.addMissingNodes = (dataObjs, enforceOrder = false) => {
    if (!dataObjs || !dataObjs.length) {
      return;
    }

    for (let index = 0; index < dataObjs.length; ++index) {
      const dobj = dataObjs[index];
      const node = publicAPI.addMissingNode(dobj);
      if (
        enforceOrder &&
        node !== undefined &&
        model.children[index] !== node
      ) {
        for (let i = index + 1; i < model.children.length; ++i) {
          if (model.children[i] === node) {
            model.children.splice(i, 1);
            model.children.splice(index, 0, node);
            break;
          }
        }
      }
    }
  };

I meant something like that.(enforceOrder could be set to true only if there are Actor2D in the scene)

This has the advantage to:

• never delete nodes to recreate them
• has a small overhead ONLY when order is changed, otherwise it is almost a no-op.

[floryst]

Good point @finetjul. I'll fix that. Meanwhile, do you know what's up with the CI?

dunno. @floryst ?

CI should now be fixed, so rebasing this PR will do the trick.

[sankhesh]

@finetjul Here's what the code from my changeset does now:

https://github.com/Kitware/vtk-js/pull/3209/files#diff-957a6188d1642e3d29c27ab761a8f827cdeff837b935eb3768a7aaba4b5deddcR27-R35

      // Force all actor2D instances to be sorted and re-pushed as scenegraph nodes.
      // This ensures that they are rendered in the correct order.
      const actors2D = model.renderable.getActors2D();
      actors2D.forEach((ac) => {
        const vn = publicAPI.getViewNodeFor(ac);
        if (vn !== undefined) {
          publicAPI.removeNode(vn);
        }
      });
      publicAPI.addMissingNodes(model.renderable.getViewPropsWithNestedProps());

Since the removeNode is inside the actors2D.forEach, it is only invoked if there are actor2D instances in the child nodes i.e. the delete, sort and recreation only happens if there are 2D actors in the view. So, unless I missed something, my code already has the advantages you listed.

[finetjul]

Since the removeNode is inside the actors2D.forEach, it is only invoked if there are actor2D instances in the child nodes i.e. the delete, sort and recreation only happens if there are 2D actors in the view. So, unless I missed something, my code already has the advantages you listed.

The problem I see is that by deleting the nodes (removeNode) to recreate them just after (addMissingNode):

• minor: vtkOpenGLActor is deleted as well as vtkOpenGLXXXMapper that free the associated GRAM (removeNode)
• minor: newInstance() is called on vtkOpenGLActor (newInstance() is not cheap in VTK.js) (addMissingNode)
• minor: new textures need to be recreated (if any) (vtkOpenGLActor::buildPass)
• major: all the data associated to the mapper must be rebuilt and re-uploaded on the GPU . (VBO creation is done on the CPU and is not cheap) (vtkOpenGLActor::buildPass)

While this could be challenged whether the performance impact is acceptable or not (it's just for vtkActor2D). It is just that it is not "required" to do all of these deletions and creations.

[finetjul]

LGTM

[github-actions]

🎉 This PR is included in version 32.13.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀