1st party WASM bindings are now available for Alpha testing

[yuvaltassa]

google-deepmind/mujoco@4086261
google-deepmind/mujoco#2585 (comment)

[zalo]

Sweet!

[zalo]

Looked around a bit; it seems like a solid start 💪

Do you think it's at the point where it can load custom models and textures from the virtual filesystem?

I'm glad that the genned header has my beloved typed_memory_views... that should make the incremental porting effort much easier while I wrap my head around all of the newly exposed public functions... it would be nice if the typescript kept the typing for the arrays...

Perhaps I can have Claude take a stab at doing the port first 😅

[zalo]

@MatiasManevi @okmatija @sebasnorena-lgtm @kbayes @yuvaltassa
Thank you for putting this WASM build together! I spent a bit porting my demo suite to use your new official WASM bindings: https://raw.githack.com/zalo/mujoco_wasm/use-official-wasm/index.html

A few versions ago, it seems like the way textures were stored and mapped from materials changed and I haven't been able to parse them since... With the new binding, I've ported most of the logic over to use the new accessors... but I still can't seem to find a way to access the textures properly 😅

Would it be possible to make it easier to access textures and uvs from materials in the official bindings?

[yuvaltassa]

I think the only meaningful change is that mat_textid is now a matrix, one row vector per material

int*      mat_texid;            // indices of textures; -1: none            (nmat x mjNTEXROLE)

So each material can have multiple textures associated with it. You can identify the textures by the nonnegative index value, and its role by its location, where each column corresponds to one of the allowed mjtTextureRole roles.

If that explanation doesn't entirely clear things up, can you write what you are trying that doesn't work?

cc @haroonq @okmatija @erez-tom

[lshdlut]

Haha, perfect timing — we just finished hammering together our own little WASM build of MuJoCo, and then the official one showed up 😅
Feels like we were still sanding the table when the real carpenters walked in.

Our repo mujoco-wasm-forge focuses on keeping a full C ABI in WASM, auto-exporting functions from headers, and syncing versioned builds (3.2.x–3.3.x) through a reproducible toolchain.
Really excited to see an official path forming — we’ll test compatibility later and can share notes if it helps the rollout.

[zalo]

I think the only meaningful change is that mat_textid is now a matrix, one row vector per material

Aha! That fixed texture loading! However, it seems like there is one last regression in my demo suite related to a change in how the mesh normals and UVs are indexed(?)

If you click into the Cassie demo here: https://raw.githack.com/zalo/mujoco_wasm/use-official-wasm/index.html
It looks like the model's UVs and Normals are messed up... This was true in my own custom MuJoCo 3.4.X bindings, so it seems like some engine change since MuJoCo 2.3.X...

When I print out the information, it looks like the normals are in the correct range, so I think they're the correct datatype... and the vertices appear to be forming correct/proper triangles, so those aren't corrupt... And this time, I've checked mjmodel.h to ensure there isn't any formatting change that I didn't expect... What could be going on @yuvaltassa ? Could it be something like the normals and UVs are now "All X's, then All Y's, then All Z's..."?

@lshdlut Your demo isn't public! https://github.com/lshdlut/mujoco-wasm-play

[yuvaltassa]

The only relevant commit I can think of is google-deepmind/mujoco@9756ed0 , does that seem relevant to you?
Perhaps @quagla can shed some light, he made the change.

We might need an MRE to diagnose this...

[MatiasManevi]

Could it be something like the normals and UVs are now "All X's, then All Y's, then All Z's..."?

It is worth mentioning that the documentation to explain these sort of things is something we are still working on. Our Javascript bindings expose underlying matrix in a very specific way.

When a function from the MuJoCo API returns a matrix (or needs a matrix as input), these are represented in the JavaScript side as flat, one-dimensional TypedArray arrays. The elements are stored in row-major order.

For example, a 3x10 matrix will be returned as a flat array with 30 elements. The first 10 elements represent the first row, the next 10 represent the second row, and so on.

Example: Accessing an element at (row, col)

// A 3x10 matrix is stored as a flat array.
const matrix: Float64Array = ...;
const nRows = 3;
const nCols = 10;

// To access the element at row `i` and column `j`:
const element = matrix[i * nCols + j];

[zalo]

The only relevant commit I can think of is google-deepmind/mujoco@9756ed0 , does that seem relevant to you? Perhaps @quagla can shed some light, he made the change.

Aha! This was it! I needed to use the mesh_facetexcoord and mesh_facenormal arrays as another layer of indirection to map UVs and Normals back to vertex indices (which is how three.js indexes meshes). 0b76374

Since my integration is now at full feature parity using these "Alpha" WASM bindings, I've merged them to the main branch 😎
https://zalo.github.io/mujoco_wasm/

Perhaps we can get some of the new control policy demos coming out of these various places integrated as a fun new online demo... would something like this be the best candidate? https://github.com/mujocolab/g1_spinkick_example

If MuJoCo itself included an evaluator or some kind of purpose-built interface for .onnx policies, that would probably make it easier to integrate demos and deploy MPCs 😅

Likewise, you're welcome to use/pick apart everything in this repo for the official MuJoCo example (which I'm sure will evolve over time to support most researchers' needs 😄).

[yuvaltassa]

cc @kevinzakka

[ttktjmt]

@zalo Wow! This looks like a perfect fit for a project I've been developing over the past few months: https://github.com/ttktjmt/muwanx

Although I haven’t implemented full ONNX model validation yet, it already supports online inference and execution!

In addition, the system can automatically collect mujoco assets and resolve file paths on the WASM's VFS, requiring only the root MuJoCo model as input.
With this, visualization of many models, including those from mujoco_menagerie, is already working successfully!

I’m also currently collaborating with the MyoSuite team, working on integrating reinforcement learning models for musculoskeletal simulations.

I also plan to make it directly callable from mjlab, since mjlab already manages observation/action info in python.
This would make it possible to upload trained models as live demos directly to GitHub Pages right after training.

[zalo]

Cool! That looks like a great usage for it! The latest version should fix the issues with normals and textures (like on the Cassie robot), and I’ve improved the visuals a little too 😄