SMPL-(H/X) model loading crash

[Gara-Dorta]

I want to load a custom SMPL-X sequence, this is the code that I have so far:

   smplx_layer = SMPLLayer(model_type="smplx", device=C.device)

    smpl_seq = SMPLSequence.from_npz(
        file="data.npz",
        smpl_layer=smplx_layer,
        poses_left_hand=data["poses_body"][:, poses_left_start:poses_left_end],
        poses_right_hand=data["poses_body"][:, poses_right_start:],
    )

when I run it, I get an error

Traceback (most recent call last):
  File "aitviewer/examples/load_smplx.py", line 32, in <module>
    smpl_seq = SMPLSequence.from_npz(
  File "aitviewer/aitviewer/renderables/smpl.py", line 327, in from_npz
    return cls(
  File "aitviewer/aitviewer/renderables/smpl.py", line 161, in __init__
    global_oris = local_to_global(
  File "aitviewer/aitviewer/utils/utils.py", line 342, in local_to_global
    if parents[j] < 0:
  IndexError: index 22 is out of bounds for axis 0 with size 22

While debugging it I tried the example/missing_frames.py, this very simple t_pose

    smplx_male = SMPLSequence.t_pose(
        SMPLLayer(model_type="smpl", gender="male", device=C.device),
        name="SMPL",
        position=np.array((-1.5, 0, 0)),
    )

For either, I get a

  File "/home/era/code/aitviewer/aitviewer/renderables/smpl.py", line 162, in __init__
    torch.cat([self.poses_root, self.poses_body, self.poses_left_hand, self.poses_right_hand], dim=-1),
TypeError: expected Tensor as element 2 in argument 0, but got NoneType

because the poses_left and poses_right hands are None, which is why I had to add them in my example above.

I also tried load_AMASS example and I get the same index 22 is out of bounds for axis 0 with size 22 error.

[Liar-zzy]

I've got the same problem, have you solved it?

[Liar-zzy]

Hi, I solved it. You can just use pip install aitviewer to install this package, do not use git clone.
There may be some problems in current code.

[azadef]

I downgraded the package to 1.12.0, and it worked.

[Liar-zzy]

I downgraded the package to 1.12.0, and it worked.

1.13.0 works as well

[Gara-Dorta]

Thanks, I solved it by downgrading to 1.13.0 and by manually loading the weights from the npz file. I was following the convention from the SMPL Blender addon but the one here is different, here is the final script that I'm using

smplx_layer = SMPLLayer(model_type="smplx", gender="neutral", device=C.device)

poses_body_end = 22 * 3
poses_left_hand_start = 25 * 3
poses_left_hand_end = 40 * 3
poses_right_hand_start = poses_left_hand_end

data = np.load(input_path)

SMPLSequence(
    smpl_layer=smplx_layer,
    poses_root=data["poses"][:, :3],
    poses_body=data["poses"][:, 3:poses_body_end],
    betas=data["betas"],
    trans=data["trans"],
    poses_left_hand=data["poses"][:, poses_left_hand_start:poses_left_hand_end],
    poses_right_hand=data["poses"][:, poses_right_hand_start:],
)

Still, it would be great if this worked on the main branch.

[Jatinkalal]

Hi, i have a small doubt
please correct me if Iam wrong

basically the SMPL-H motions from AMASS has poses of shape N x 156
i think 156 for 52 joints considering axis angle rep for each

so is it that the hand parameters (left hand + right hand) are 30? first 22 of 52 is body + root while next 15 are left hand and next 15 are right hand?
Thank you

[MarilynKeller]

The problem arise because the whole skeleton (kinematic tree) is not passed. SMPLH has more than 22 joints so parents[22] should return a valid joint.

I believe the issue comes from here:

aitviewer/aitviewer/renderables/smpl.py

Line 430 in 8fb6d46

skeleton = (

        skeleton = (
            self.smpl_layer.skeletons()["body"].T
            if not self.smpl_layer.model_type == "mano"
            else self.smpl_layer.skeletons()["all"].T
        )

The in the not MANO case, the whole skeleton should be passed, I replaced these lines with:

skeleton = (self.smpl_layer.skeletons()["all"].T)

Which fixes it.

[MarilynKeller]

I got the underlying issue now.

The MANO commit changed this:

https://github.com/MarilynKeller/aitviewer/blob/8fb6d4661303579ef04b3bf63ac907dbaecff2ff/aitviewer/renderables/smpl.py#L162C71-L162C86

In the previous version,only the body pose parameters would be passed here, not the hands. skeleton only contains the body skeleton, hence the crash.

Here is the pull request that fixes it: #80

[kaufManu]

The PR #80 might need some time to be merged, but this problem got resolved with PR #83 as well, which is merged on main. I've used this dummy motion to test it:

import numpy as np

from aitviewer.configuration import CONFIG as C
from aitviewer.models.smpl import SMPLLayer
from aitviewer.renderables.smpl import SMPLSequence
from aitviewer.viewer import Viewer

if __name__ == "__main__":
    smplx_layer = SMPLLayer(model_type="smplx", gender="neutral", device=C.device)

    poses_body_end = 22 * 3
    poses_left_hand_start = 25 * 3
    poses_left_hand_end = 40 * 3
    poses_right_hand_start = poses_left_hand_end

    N = 120  # number of frames
    # SMPL-X has 55 joints * 3 = 165 pose parameters total
    poses = np.zeros((N, 165))
    betas = np.zeros(10)
    trans = np.zeros((N, 3))

    t = np.linspace(0, 2 * np.pi, N)

    # Sway torso left/right (root rotation around Z)
    poses[:, 2] = np.sin(t) * 0.15

    # Turn right arm: body joint 16 = right_shoulder
    #                 body joint 18 = right_elbow
    poses[:, 51] = np.sin(t) * 0.6
    poses[:, 57] = np.sin(t + np.pi / 4) * 0.9

    # Move forward
    trans[:, 2] = t / (2 * np.pi) * 2

    seq = SMPLSequence(
        smpl_layer=smplx_layer,
        poses_root=poses[:, :3],
        poses_body=poses[:, 3:poses_body_end],
        betas=betas,
        trans=trans,
        poses_left_hand=poses[:, poses_left_hand_start:poses_left_hand_end],
        poses_right_hand=poses[:, poses_right_hand_start:],
    )

    v = Viewer()
    v.scene.add(seq)
    v.run()

Can you confirm that this works as intended?

[kaufManu]

Fixed in release v1.14.0.