[EXTRACTOR ERROR] Bone Index out of range of Remap List

[ATH4]

Description
runtime error during normal usage.

System info:

• Filediver version: 0.3.3
• OS: Windows

Error output
Only include errors that actually pertain to the described type of file.

STATUS File 617/27607: 0x0991016be957eb39.unitpanic: runtime error: index out of range [198] with length 1
...

Additional context
full log:
./filediver -c "model:format=glb"
INFO Using game found at: "C:\Program Files (x86)\Steam\steamapps\common\Helldivers 2"
INFO Output directory: "extracted"
INFO Known hashes: names 23.47%, types 95.45%
INFO Extracting files...
ERROR extract 0x003d21b411d799d4.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x004865a7e5da5b2c.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x005c2262f672caf4.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x00bcf8247a544cdc.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: [2]float16
ERROR extract 0x00f84088090c623c.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x01144a44b65471bd.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x01b741a724fd9bb3.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x01d8f262f56227fb.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x029cfc4c193211c3.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x02c978264cdcccad.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x02ca1c5210037cae.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x02ecb3e7587540bb.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: [2]float16
ERROR extract 0x0332aa9db9f3471f.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x0364ca45cf3fb606.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x0377ea04bcbd7319.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x0387a158d04848f4.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x03e67a19b07c6523.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x0416984f4922757b.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x042592e43d4d977d.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x046ef8f635d3b937.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x0591971af87d0cba.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x068cc25ab414bb1c.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x070bc8d3e18bb8f9.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x070df80412814549.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x072837478929b018.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x079c966ee10536f9.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x07b2590feb74c102.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x07d605bd50c98461.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x07d9c4b2824dcc3d.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x07db4515d78027b6.unit: expected bone index item to have format [4]uint8, but got: 20
ERROR extract 0x07fe9ad17429dc6b.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: [2]float16
ERROR extract 0x080694e77eb97e77.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x08886d27a6f6db8c.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: [2]float16
ERROR extract 0x089159bcfcbfc957.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x08d91172be0a96ee.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: float32
ERROR extract 0x08f6089289c83d22.unit: expected bone index item to have format [4]float16 or [4]uint8, but got: [2]float16
STATUS File 617/27607: 0x0991016be957eb39.unitpanic: runtime error: index out of range [198] with length 1

goroutine 1 [running]:
github.com/xypwn/filediver/extractor/unit.remapMeshBones(...)
/github/workspace/extractor/unit/extractor.go:185
github.com/xypwn/filediver/extractor/unit.ConvertOpts({0xf4cea0, 0xc0022f8540}, 0xc000027118)
/github/workspace/extractor/unit/extractor.go:453 +0x21f0
github.com/xypwn/filediver/extractor/unit.Convert({0xf4cea0, 0xc0022f8540})
/github/workspace/extractor/unit/extractor.go:557 +0x165
github.com/xypwn/filediver/app.(*App).ExtractFile(0xc00008a890, {0xf4bd30, 0xc000066050}, {{0x991016be957eb39}, {0xe0a48d0be9a7453f}}, {0xda0fa5, 0x9}, 0xc000088810, 0xc00008c078)
/github/workspace/app/app.go:431 +0x156f
main.main()
/github/workspace/cmd/filediver-cli/main.go:206 +0x1a45

[xypwn]

It seems like the bone indices are being used without bounds checking. I'll look into it once I'm at my home computer. Thanks for the report!

[xypwn]

@ryanjsims maybe you can immediately recognize what exactly went wrong here, since you wrote the code. If not, that's also fine and I'll figure it out myself :)

Relevant sections are:

Panic location:

filediver/extractor/unit/extractor.go

Line 185 in a3a11fa

remapIndex := mapping.RemapList[0][mesh.BoneIndices[i][j]]

Bone index / remap list creation:

filediver/stingray/unit/unit.go

Line 518 in a3a11fa

for i, skeletonMapOffset := range skeletonMapOffsets {

[xypwn]

Alright, I hope everything works for you now!

I turned the panic into a regular error so the program doesn't completely fail and only skips the erroneously read file. I still don't know what exactly happened (because I didn't actually write the skeleton/bone stuff, but also because we had to reverse engineer everything), but this is something that may be looked at a bit closer, along with fixing a lot more error messages, especially in model extraction.

(The additional research that has to be done is also why I'm reopening this issue)

[ryanjsims]

@xypwn Sorry, I was working on other stuff but I've gotten back to looking at the code again: it looks like the reason for the panic boils down to simply not correctly loading all the data in the unit file and making incorrect assumptions about how bones and components work.

I've been doing some more research on how the format is loaded and I've got a branch that implements a more correct unit format on my fork. It isn't perfect yet, but it resolves several incorrect assumptions I made when initially writing the skeleton export code

[xypwn]

@ryanjsims Absolutely no need to apologize! I haven't had much time recently either to work on almost anything. After all, this is voluntary work and I really appreciate you sharing your amazing contributions :)

Seems like you made some really good progress with unit files. I'll definitely check it out later and am super excited about the improvements!