Help for dissertation in BSc CS

[akonst08]

Hello All,

I hope you are well. I would like some help on an issue I have on my thesis and I should mention that my timeline is very short.
Now about my topic-concern:
I have a YOLOv11 detector trained on 8 hysteroscopic lesion classes (medical), but I now received about 20–30 videos that contain endometritis (lesion) and I do not have frame-level annotations or bounding boxes. I only know at video level that endometritis is present, and I have no clinician support to identify where it appears (specific time of the video). I need the fastest practical pipeline to mine high-probability candidate frames, generate pseudo-labels, and train an additional detection class without retraining everything from scratch. My current concern is that the 8-class detector may not detect anything in these videos, so candidate mining should not depend on the existing detector. Please propose a step-by-step, time-efficient, code-oriented workflow using anomaly ranking, temporal consistency, SAM-assisted region proposals, and iterative pseudo-label filtering.
My dissertation won't be published, however is an important matter that would lead to my graduation. I spent many hours, running experiments that required several hours and I had no help at all, however due to time limitation I am a bit stressed.

I would appreciate any help and advices and thanks for your time reading this!

[UltralyticsAssistant]

👋 Hello @akonst08, thank you for your interest in Ultralytics 🚀, and thanks for sharing the context around your dissertation. This is an automated response to help get things moving quickly; an Ultralytics engineer will also assist you soon 😊

We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

For your case, since this looks like a training/workflow question under time pressure, it would help a lot if you could share:
-Your current ultralytics version
-A short description of your existing 8-class detector setup
-A few sample frames or screenshots from the videos
-Any current candidate-mining or pseudo-labeling scripts you have already tried
-Training logs, errors, or output examples from your latest experiments
-If applicable, a small minimum reproducible example that demonstrates the current blocker

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community.

Upgrade

Upgrade to the latest ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8 to verify your issue is not already resolved in the latest version:

pip install -U ultralytics

Environments

YOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

-Notebooks with free GPU:
-Google Cloud Deep Learning VM. See GCP Quickstart Guide
-Amazon Deep Learning AMI. See AWS Quickstart Guide
-Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Given your timeline, the fastest practical route is weakly supervised mining rather than direct 9-class retraining: extract frames at 1-2 FPS, drop blurry and near-duplicate frames, rank candidates detector-independently with image-embedding outlier scores, keep only temporally consistent runs of 3-5 consecutive high-score frames, use SAM on those runs for region proposals, manually verify only the top few hundred boxes to build a small seed set, then fine-tune from your existing best.pt on a mixed 9-class dataset plus a small replay subset of the original 8 classes to reduce forgetting; after that, pseudo-label the remaining videos with high conf and save_txt=True, keep boxes only if they are stable across adjacent frames and pass simple size/aspect checks, retrain once more, and stop after 1-2 iterations—the closest Ultralytics reference is this semi-supervised learning guide, and if helpful I can sketch the exact folder layout and minimal train()/predict() script next.

[akonst08]

Therefore, should I use something like ResNet after extracting the frames to rank them and then proceed to SAM? I would be looking at this from tomorrow and I would come back.

Thanks for your reply and help!

[glenn-jocher]

Yes—that’s a sensible order: extract frames first, rank them with a pretrained image encoder (ResNet is fine for speed, though CLIP/DINO embeddings usually separate anomalies better), keep only short temporally consistent high-score runs, then send those frames to SAM for proposals and use the verified boxes for pseudo-labeling; the closest Ultralytics references are the semi-supervised learning guide and the active learning guide, and when you come back I can turn that into a minimal script.

[akonst08]

Instead of 20-30 videos, I was given 8 videos and a medical report indicating 19 frames that show lesion (However, they are not labeled with the exact region that the lesion appears.). I built an initial pipeline as an automated annotator, that extracts the frames from the pdf, sends them into DINO (shouldn't have done that in this anyway) and then I am using SAM to estimate the regions. Yet, I saw that SAM draws boundaries of regions very very badly. I mean (imagine the image outside of hysteroscopic view which is a total black that considers those as different regions... and the whole of hysteroscopic as a one.) I did all these to at least have a base seed of more official labeled frames before proceeding with the rest. But still I am unsure about it, how would I ensure that are more valid and formal those labels? Or should I stop thinking of that and proceed with what I got ?

Thanks again in for your time.

[glenn-jocher]

I wouldn’t treat those raw SAM masks as labels yet; for hysteroscopy, first crop or mask the circular field of view so the black border is excluded, then use SAM only for candidate regions and convert the reviewed outputs into coarse detection boxes, because a small clean seed set from those 19 positive frames is much safer than many noisy labels—the key part is annotation quality control, as covered in the data collection and annotation guide, and then you can fine-tune from your current weights and pseudo-label the remaining videos.