Train YOLOv8-pose to detect another objects

[mermetal]

Hello.
Is it possible to have the yolov8-pose model estimate a person's pose while also detecting other objects?
For example, yolov8-pose can detect people and their skeleton points while also detecting dogs, cars, books, etc.

If that's possible,
Is it possible to train a model that can already detect people and estimate posture to detect dogs, cars, books, etc.?
For example, if the pre-trained model only learns dog, car, and book detection, the new model can detect dog, car, book, and person and do pose estimation.

[pderrenger]

Hello! 😊

Yes, it's possible to have YOLOv8-pose model estimate a person's pose while also detecting other objects like dogs, cars, books, etc. You can achieve this by training the model on a dataset that includes both the pose annotations for people and bounding box annotations for other objects you're interested in detecting.

To train a model that can detect additional objects alongside pose estimation, you would start with the pre-trained YOLOv8-pose model and fine-tune it on a dataset that includes your new classes (e.g., dogs, cars, books). Ensure your dataset contains images annotated with both skeleton points for humans and bounding boxes for the other objects.

Here's a simplified example of how you might start the fine-tuning process:

yolo train data=your_dataset.yaml model=yolov8-pose.pt

In your_dataset.yaml, you'll specify paths to your training and validation data, which should include annotations for both poses and the additional objects you want to detect.

Remember, the key is in the dataset preparation. Ensure your annotations are accurate for both the poses and the additional objects.

Happy training! 🚀

[shashwatsaini]

Could you please confirm if this is what the data must look like? I have 3 classes, where class 2 corresponds to people with 17 pose points. For the other two classes I am only inputting the bounding box. I have tried finetuning yolov26n-pose, but it does not predict classes 0 and 1, and it throws errors stating it is unable to compute metrics for them.

1 0.660185 0.313542 0.008333 0.057813
2 0.711343 0.293620 0.078241 0.109115 0.68831 0.252822 0.963599 0.693005 0.249961 0.956784 0.684427 0.250431 0.825707 0.701632 0.250968 0.853394 0.682077 0.252231 0.20295 0.703995 0.26283 0.990138 0.690359 0.262732 0.921034 0.697283 0.281059 0.937616 0.690464 0.278467 0.268392 0.678109 0.290888 0.901869 0.683502 0.287566 0.281517 0.727319 0.28666 0.981636 0.715176 0.286896 0.941175 0.701931 0.310286 0.991839 0.697094 0.311807 0.964164 0.725921 0.337564 0.987465 0.701513 0.338792 0.963393
1 0.425463 0.559896 0.072222 0.284896
2 0.216898 0.345182 0.423611 0.540365 0.123865 0.212389 0.0140285 0.118958 0.210758 0.00906921 0.127765 0.201855 0.00640911 0.0842435 0.220409 0.800372 0.15851 0.200101 0.881079 0.0894174 0.27225 0.996709 0.223398 0.228319 0.997459 0.080946 0.329384 0.980066 0.317535 0.178369 0.986473 0.0832596 0.377907 0.977041 0.385854 0.118171 0.966664 0.1354 0.386532 0.997914 0.235843 0.380883 0.998749 0.190711 0.485933 0.996644 0.213215 0.500224 0.996074 0.215998 0.563143 0.991857 0.104054 0.544141 0.992079
2 0.846759 0.367839 0.305556 0.701302 0.942257 0.0917912 0.416866 0.947666 0.0772386 0.508527 0.943332 0.0795712 0.0221035 0.974015 0.0813966 0.446585 0.969979 0.0847713 0.00323663 0.984031 0.160464 0.754081 0.991169 0.151944 0.406158 0.980222 0.273019 0.947384 0.907899 0.167648 0.916348 0.89059 0.349761 0.958934 0.78124 0.1504 0.987233 0.976621 0.37761 0.514544 0.979729 0.377879 0.356599 0.967395 0.524194 0.815892 0.959195 0.520439 0.548606 0.970801 0.686224 0.659988 0.967651 0.679522 0.512681

I have attached the dataset yaml as well as the training script.

detection_training_data.yaml
run_pose_estimator.py

[glenn-jocher]

No — that label layout is not a supported Ultralytics YOLO pose format: in a pose dataset, every object row must follow the same keypoint schema defined by one global kpt_shape, so mixing bbox-only rows for classes 0/1 with 17-keypoint rows for class 2 will cause the missing metrics/predictions you’re seeing. For this use case, the practical path is a detect model for all classes plus a separate person-only pose model, unless you want to custom-modify the training/validation code for per-class keypoints; after relabeling, delete any labels/*.cache files before retraining.