Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize spring bones computations #1539

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

mrxz
Copy link
Contributor

@mrxz mrxz commented Nov 25, 2024

Spring bones tend to be quite performance intensive on the CPU side. I looked into ways to optimize them, while retaining their original behaviour. This PR consists of three commits each focussing on a different area.

See the following table which shows the average duration a call to vrm.update took. The changes are stacked, so the Collider shape offset column contains all three optimizations.

Avatar Before PR Pre-compute order World space computations Collider shape offset
VRM1_Constraint_Twist_Sample.vrm 658.0 µs 179.1 µs 164.3 µs 145.7 µs
AvatarSample_C.vrm 1.0 ms 615.3 µs 473.8 µs 415.2 µs
Zonko_VRM_221128_ps.vrm 3.2 ms 318.0 µs 269.1 µs 200.1 µs
AKAI Original VRM by Shugan.vrm 161.7 µs 105.8 µs 92.4 µs 95.1 µs

Pre-computing spring joint update order (a21e474)

The order in which spring bones need to be updated can be pre-computed making the actual update a lot more straightforward and performant. Whenever a spring bone is added or removed, the order is recomputed. The following things are precomputed:

  • The order in which the spring bones should be updated when taking all dependencies into account (_sortedJoints)
  • The ancestors from the lowest common ancestor of all the spring bone roots to the spring bone roots (_ancestors)

Computing stiffness and gravity in world space (2e14729)

The VRMSpringBoneJoint.update method converted all forces into center space. This is required for inertia, but for both stiffness and gravity this isn't strictly needed. It saves a couple of matrix multiplications by handling these in world space instead.

Additionally some of the vector math could be simplified (e.g. .add(v3.multiplyScalar(scalar)) -> .addScaledVector(v3, scalar)) slightly reducing the amount of operations required.

Aligning collider position with collider shape offset (7449a0d)

The VRMSpringBoneCollider is a THREE.Object3D with an identity transform. The collision shapes contain an offset, which is used to compute the world position of the collider shape. But this is done each time calculateCollision is called, which can be many times per-frame, despite the actual world position of the collider not changing in between.

By setting the position of VRMSpringBoneCollider to the offset, the world position can be retrieved from the worldMatrix instead, ensuring it's only calculated once per collider per update. To further reduce the number of calculations done, the output vector (target) is now only updated in case the distance is negative.

This did require some changes to the unit tests, as they don't use VRMSpringBoneCollider but test against the VRMSpringBoneColliderShape directly.

Notes

For benchmarking and testing I created a setup where I loaded the VRM, called vrm.update(FIXED_DELTA) and compared the world positions and rotations of all joints to reference values based on running the same on dev. So for the cases and VRM models tested above, the output after one update was identical (with EPSILON=0.0001).

/**
* List of dependencies that need to be updated before this joint.
*/
public get dependencies(): THREE.Object3D[] {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SHOULD] Existing VRMConstraints already have signatures get dependencies(): Set<THREE.Object3D>. I want to align the interface unless it needs to be an array instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain why you moved this to the public property though? I don't think it's a bad idea, but it is still called only from VRMSpringBoneManager.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I moved it as I wanted to cache the list of dependencies, since it caused a lot of memory allocations in the hot-path (VMRSpringBoneManager.update). The VRMSpringBoneJoint itself seemed like a good place to keep the cached list as it could also invalidate it when needed. But after pre-computing the joint order, it was no longer in the hot-path, so I removed the caching again.

In the end the move wasn't needed. But I like the consistency with the VRMConstraints, so I'll update the signature to return a Set<THREE.Object3D> as well.

// it is required when the spring bone chain is sparse
traverseChildrenUntilConditionMet(springBone.bone, (object) => {
const lca = lowestCommonAncestor(ancestors);
this._ancestors = [];
Copy link
Contributor

@0b5vr 0b5vr Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure this this._ancestors has to be an array since the 0th element should calculate ancestors' world matrices.

[SHOULD] However, the reason why the variable this._ancestors is needed is kinda unintuitive. It would be very appreciated if you give them a proper doc comment to help future us.
I believe it goes like "A list of ancestors of the entire SpringBone joints. Their world matrices have to be calculated before we update the entire SpringBone system. The first element is ~~"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, the _ancestors list is ordered such that their world matrices can be updated correctly. The first element is the lowest common ancestor for which we also need to make sure its ancestors have their world matrix updated.

The graph below is an example of a Scene graph and shows which nodes would be in the _ancestors list in that case:

graph TD
    S[Scene] --> AP[Object3D]
    S[Scene] --> B[Object3D]
    AP --> A[Object3D<br>ancestors#91;0#93;]
    A --> C[Object3D<br>ancestors#91;1#93;]
    A --> D[Object3D<br>ancestors#91;2#93;]
    C --> S1[Springbone 1]
    D --> S2[Springbone 2]
    D --> S3[Springbone 3]
  classDef ancestor fill:#AFA,stroke:gray
  classDef lancestor fill:#4D4,stroke:black
  classDef o3d fill:#EEE,stroke:gray
  class C ancestor
  class D ancestor
  class A lancestor
  class S o3d
  class AP o3d
  class B o3d
Loading

I've added a doc comment that should hopefully clarify things for future readers.

@0b5vr
Copy link
Contributor

0b5vr commented Nov 26, 2024

I reviewed the first two commits.
I've left several comments on the first one. I might miss the point so feel free to correct them.
The second one looks perfect to me.

I will review the third one tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants