Should prepareRay do scene.updateMatrixWorld()?

[dangrabcad]

After the React reconciliation, the scene-graph is in an inconsistent state: objects have had their position/rotation/scale properties set from React props, but their matrices will not be updated until the next WebGLRenderer.render (which happens from the requestAnimationFrame callback). At this time, if a pointer event is delivered to the application, the ray will be cast against the wrong objects, because the scene-graph is not yet ready.

In my application, this causes a problem: sometimes an operation might add (say) 1000 new objects to the scene-graph as a result of an event handler (a button click or keypress). If the user moves the pointer during this operation, then Chrome will deliver the pointer event as soon as the previous event finishes processing, without calling the rAF callback and giving the application a chance to draw. This means that all 1000 new objects will still have the identity matrix when raycasting starts. This makes the raycasting take over a minute when it normally takes a few milliseconds, because all 1000 objects are in the same place and all of them do the full triangle-intersection code (whereas they're usually eliminated quickly by the bounding-sphere check).

Should prepareRay (or something else in events.ts) have the same if ( scene.autoUpdate === true ) scene.updateMatrixWorld() as WebGLRenderer.render, to ensure that the scene-graph has up-to-date transforms before ray-casting? Although my application has an extreme case of the problem - lots of objects being added in a single, long operation that gives plenty of time for a pointer event - it could potentially happen in any application and cause confusing events.

[CodyJasonBennett]

Will have to cc @drcmda, noticed the same. This is also present in hooks like useFrame where the first invocation won't have updated matrices.

[drcmda]

you can finally exchange the full compute, this would fix it:

<Canvas onCreated={state => {
  state.setEvents({ compute: (e, { pointer, size, raycaster, camera, scene }) => {
    scene.updateMatrixWorld()
    pointer.set((e.offsetX / size.width) * 2 - 1, -(e.offsetY / size.height) * 2 + 1)
    raycaster.setFromCamera(pointer, camera)

it would happen once per event.

but it seems costly, we also have to do this for drei/instances and it's something that really bothers me because it creates additional overhead just to swat a rare first-load issue.

i think the root cause is something that could be fixed, i would rather try to take care of first load updateMatrixWorld after all objects have been created. there must be something ...

one quick fix would be this for instance:

<Suspense fallback={<KillEvents />}>
  <ThousandModels />
</Suspense>

function KillEvents() {
  const setEvents = useThree(state => state.setEvents)
  useEffect(() => {
    setEvents({ enabled: false })
    return () => setEvents({ enabled: true })
  }, [])
}

ps this is v8 api. the latter would also be possible with v7, you would mutate raycaster.enabled instead.

pps. wouldn't this also be a great usecase for drei/Preload?

<Suspense fallback={<KillEvents />}>
  <ThousandModels />
  <Preload all />
</Suspense>

it would be wonderful if it could also take care of that.

[CodyJasonBennett]

Could be possible in onCreated but objects that are mounted later would have the same problem as before.

[drcmda]

this exchanges the compute function though, meaning that for every pointer it will first generate matrices, no matter if objects come now or later. but that's a user land fix of course.

[dangrabcad]

I like the suggestion of doing it after the reconciler applies props instead of before raycasting. There would still be a burden for people mutating their scene-graphs directly with imperative code, but I could live with that in my application. (I guess it wouldn't be a problem for useFrame if that happens in the same rAF callback as render, and it could be done after all the useEffect callbacks are called so they needn't be a problem either.)

That said, I don't think the overhead is a massive problem. If you're using the default rAF frameloop instead of invalidate, this function is getting called at 60 fps even if React is not doing anything.