Osim with neck, scapula and arms.

[TKojar]

Hi, I am trying to build an osim file that contains neck, scapula and arms. The Rajagopal and Ajay (https://simtk.org/projects/scapulothoracic) were combined in the Skel bsm.osim file https://skel.is.tue.mpg.de/, but it only works in nimblephysics not opensim.

So I am trying build the analogous bsm.osim that will function in Opensim. Instead of eyeballing, I want to tranfer the precise anatomical numbers from Ajay into bsm.osim (i.e. I removed the shoulder parts from bsm which were only compatible for nimblephysics).

For example, the following is from Ajay:

<parent_body>thorax</parent_body>

<location_in_parent>-0.02 -0.0173 0.07</location_in_parent>

<orientation_in_parent>0 -0.87 0</orientation_in_parent>

-0.05982 -0.03904 -0.056

-0.5181 -1.1416 -0.2854

How do I transfer this information in the modern opensim format in terms of physicalloffsetframe? Here are my latest changes: https://github.com/TKojar/Opensim/blob/main/bsm.osim

Is there a way to use the Opensim-creator software to achieve this?

alternatively, do you know if the Osim file I am looking for has already being built by someone? Thank you.

[adamkewley]

I don't know if the osim file you're looking for has already been built by someone, but I'm no expert. I'd recommend trawling through https://simtk.org/ with search terms like "Upper Extremity", "Scapulothroacic", etc. - you might get lucky

---

I can't visualize your model (I'm missing the mesh files, etc.) but, from what I can see of the XML and frame visualization, I think what you're already doing is roughly correct. The way I'd do it is to either copy the entire kinematic chain XML from the shoulder downwards (i.e. the scapula joint, upper arm, lower arm, hand, etc.) OR replace the existing shoulder joint definition (if available) with a ScapulothoracicJoint and then ensure the various parameters are correct.

The problem you might run into, and I think what you are asking for when referring to PhysicalOffsetFrame (POF), is how you ensure the splicing of the model is in a geometrically-correct location. The main considerations are:

• The joint definition you pasted contains fields like location_in_parent and orientation_in_parent. These are the "legacy" way of defining joint offsets. For backwards compatibility, they still work, but the "modern" way to handle them is to put the numbers into a PhysicalOffsetFrame in the <frames> element of the Joint (they can actually be stored anywhere in the model, but they're usually stored in <frames>). "Modern" joints have <socket_parent_frame> and <socket_child_frame> properties that connect the joint to those frames. I think you have already ported these numbers?
• The position/orientation of those frames is strongly dependent on what they are attached to (socket_parent). E.g. in the source model it looks like they were defined w.r.t. the thorax body. When you are splicing the model, it might be that the new target model doesn't have a thorax, or the origin of thorax is defined differently.
• If the parent frames are defined differently, then you have a fundamental challenge that needs to be solved. You need to figure out how the two coordinate systems map onto each other. This isn't something that can be automated because it's strongly dependent on how the frames were originally defined in both models. It might be that you need to add a custom PhysicalOffsetFrame or StationDefinedFrame into your target model that has the same definition as thorax in the source model, and then you can attach the existing joint to that. There's documentation pages in OSC that explain how to add frames to a model (e.g. https://docs.opensimcreator.com/manual/en/latest/make-a-leg.html does it with StationDefinedFrames).

---

OSC doesn't have visual/automated tools for splicing models together, only editing tools that make building/modifying model files easier. It's quite rough to automate splicing in a UI because there's both technical/datastructure concerns (e.g. name collisions, socket rewiring) and biomechanical/definition concerns (e.g. how the coordinate systems map onto each other, if splicing entails attaching two models rather than having two kinematically separate models in one scene).

[TKojar]

Geometry.zip

https://github.com/TKojar/Opensim/blob/main/bsm.osim

Hi I added the osim file and the geometry now. I combined the Seth model with the bsm.osim file by Keller et all. Can you take a look?

My mesh file is a bit off but by computing the opensim-transforms for this file versus the Seth model for shoulder, I got the same ones, so I hope the kinematics were successfully preserved.

Any comments/feedback will be very helpful.

Some questions:

1)main question: any advice on how to build the left-side? can I use opensim-creator to reflect?

2))can I use opensim-creator to fix that mesh misalignment?

3)is there a way to use opensim-creator to more carefully rebuild this from the two models?

[adamkewley]

Hi @TKojar , sorry, I was away for Christmas and I am currently a little bit preoccupied with other tasks (maybe someone else in the community could help me out here 😉 )

1. You can, in principle, use opensim-creator to reflect an object. I've seen researchers do it by scaling meshes by a negative quantity around the reflection plane. E.g. if you scale X by -1 you are effectively reflecting along the X axis (anything above ends up below, and visa versa). This might behave bizzarely in some situations, though, because (e.g.) meshes aren't necessarily robust to that kind of transform (it flips the surface normals). The alternative is to use third-party software.
   
   Reflecting not-meshes in the model (e.g. muscle points) is strongly dependent on how you are handling the "reflection" of the reference frames. In OpenSim, reference frames are always right-handed. This cannot be avoided. So you can't "reflect" a reference frame (it would become left-handed), and this means that you must make decisions around how each axis is defined (or use a standardized frame definition, like ISB's, described in OSC's Station Defined Frame documentation).
   
   This is all to say, reflecting stuff in OpenSim is harder than something like (e.g.) Blender because it's an articulated model with joint definitions that are strictly anatomically relevant. It's hard to handle it with a generic UI tool without that tool asking a bunch of very specific geometric questions via popups etc.
2. You can use opensim-creator to realign/move offset frames, which meshes are usually attached to, which could be used to fix your mesh alignment (in principle). If you just want to eyeball it, then that's easy enough. OSC also has tools for adding new offset frames and reassigning sockets between meshes/frames (e.g. reassign the mesh to a new frame).
   
   However, OSC isn't magical - it can't automatically figure out what you want. It might seem simple to (e.g.) want two meshes to "align well", but what you are actually asking for, algorithmically speaking, is "A spatial transform of one mesh such that its volume maximally intersects with the volume of some other mesh, where each mesh is a point cloud of triangles, potentially with a different triangle density". This might be quite computationally expensive to figure out in the GUI as a user drags the mesh around. It might also be an invalid algorithm (maybe you know a better way to achieve what you want). Therefore, for those kinds of problems, I usually recommend the user writes a python script or similar to calculate the relevant orientation/translation, but it isn't something that OSC can automatically do right now (I can imagine a future scenario where it's a contextual action on right-click, but it's quite specific/expensive and isn't currently in OSC).
3. OSC doesn't have tools for combining two models. It's something users have repeatably asked for, but it hasn't been done because it's very difficult to actually design a GUI that does it robustly and with enough customization points to enable the user to actually express what they want when they frankenstein two models together. It's hard to do right, and things like scaling, reattaching parts of the model, renaming components, etc. are decisions that the GUI must defer to the user (I can't automatically figure these things out for you).

[TKojar]

Here is another version: https://github.com/TKojar/Opensim/blob/main/bsm_Seth_v2

I kept the mesh of the thorax by Seth and so it looks off for neck and lumbar. Any advice on how to fix this?

• 4)In the above geometry folder, I have the thorax mesh by Seth and BSM. I see that in the opensim creator, I have access to the mesh-frame coordinate frame. Any advice on how to adjust the mesh-frames so that the above picture looks normal? It is tricky because I am trying to combine the mesh-files from two different model files.

• 5)I was thinking to try to use the anatomical markers from Seth's file and the BSM/Rajagopal to help me figure out how to position the coordinate mesh frame.

• 6)Is there a way to verify that my inverse kinematics by Seth are preserved? I simply did some motions with my IMUs and compared the angles by the two models. I also used the opensim "get ground transform" function to compare them.

[adamkewley]

4. You can add offset frames to any existing body/offset frame in the model (right-click, add, offset frame). The properties editors for offset frames lets you select which frame to edit the property with respect to (e.g. ground, or some other frame) and you can also reattach offset frames to other frames without moving the attached objects via the "Re-express in chosen frame" option from the sockets menu where you can reassign the parent of an offset frame. This is essentially almost everything that a GUI like OSC can provide in terms of orientation/translation tooling for frames in a model. Anything more is entering "solver" territory (e.g. least squares distance between meshes, points in the model, etc.) and different fitting techniques require different solvers, so OSC doesn't provide that (yet).
   
   You should ideally be asking why you are seeing misalignment in the first place. It's usually because a frame definition in your model isn't anatomically-based, and was just some orientation/translation value in an offset frame, rather than based on (e.g.) landmarks on bones. The StationDefinedFrame component is specifically designed to tackle those kinds of problems because, otherwise, you are playing the realignment game on arbitrary mesh data.
5. That's exactly how they probably placed the PhysicalOffsetFrame in the original model, yes. The StationDefinedFrame is a newer component that isn't fully supported by OpenSim GUI yet (because their release cycle is currently 2 years - wtf), but it is how frames should be defined in models moving forward. You might be able to figure out which landmarks they used to define the PhysicalOffsetFrame in the original model and then place some stations in your new model and define a StationDefinedFrame based on that (this is done extensively in the later tutorials in OpenSim Creator's documentation). The original sin that you are trying to work around here is that the original model doesn't encode key pieces of information (such as frame-to-landmark relationships) that are necessary when trying to reattach parts of one model to some other model.
6. I don't know what you mean by this. Do you mean "if I used the same marker locations as originally used by AS with my new model, that they should produce the same joint angles" (unlikely) or "if I use new IMU measurements with my new model they should produce the same joint angles"? No idea. You have one set of data from one experiment + model (didn't read, but I'm assuming it's markers with the Seth model), and you have a separate set of data that's applicable to your new model (IMU + new model). I'm not a researcher (at all), but as "some guy on the internet", it feels like your validation procedure should validate your new model against your new measurement technique because you have Frankensteined parts of a model together and made custom modifications, so direct comparisons with the Seth research should probably take the form of "has similar R^2 fits from IK", "behaves similarly in these ways" - they aren't the same model, it isn't the same data.