Hyperparameter Optimisation: Raytune vs no Raytune

[jingyu-itmax]

I want to know what is the difference when we call the tune function in ultralytics, with and without raytune. Does it use the same genetic-mutation algorithm, but just runs in parallel with raytune? Or does it use Bayesian optimisation when raytune is enabled? I want to know how does ultralytics do hyperparameter optimisation, if raytune is faster why is it just not enabled defaultly.

[UltralyticsAssistant]

👋 Hello @jingyu-itmax, thank you for your interest in Ultralytics 🚀! This is an automated response to help you get started quickly—an Ultralytics engineer will also assist soon 🙂

We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question (including hyperparameter tuning questions like Ray Tune vs non-Ray runs), please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

To help us answer your Ray Tune vs non-Ray Tune question precisely, please share:
-Your ultralytics version (pip show ultralytics)
-Your OS + Python + PyTorch versions
-The exact command or Python snippet you’re using to call tune
-Whether you’re running single GPU, multi-GPU, or multi-node
-Any logs/output that show the tuning strategy/config being applied

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community.

Upgrade

Upgrade to the latest ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8 to verify your issue is not already resolved in the latest version:

pip install -U ultralytics

Environments

YOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

With model.tune(use_ray=False) Ultralytics runs the built-in Tuner evolutionary search (genetic-style mutation of hypers across iterations, i.e. sequential “evolution” and it writes the best result to best_hyperparameters.yaml), while model.tune(use_ray=True) switches to the Ray Tune integration (run_ray_tune) which samples from a defined search space and uses Ray Tune’s scheduling/parallel execution (ASHA early-stopping by default) to run multiple independent training trials concurrently when you have the resources; it’s not doing Bayesian optimization unless you explicitly configure a Ray search algorithm yourself (our wrapper defaults to basic sampling + ASHA). See the Model.tune() reference and the Ray Tune integration docs for the exact behavior and parameters. Ray isn’t the default mainly because it adds a heavy optional dependency/runtime and only tends to be “faster” when you can actually parallelize trials (multi-GPU or a cluster), whereas the built-in tuner is lightweight and works out-of-the-box.

[jingyu-itmax]

Thank you for the reply. So, if raytune=False, this would be ideal for someone running experiments/projects with limited GPU resources, iterations should be low? Otherwise it will take them a long time to get the best params (unless they have the time). And with raytune, it would benefit if we set higher iterations because it is parallelisable with a large gpu cluster?

[glenn-jocher]

Yes—treat iterations as “number of full training trials,” so on a single/limited GPU you’ll usually want to keep iterations (and often epochs/imgsz) modest to stay within your time budget, then retrain once with the best settings; Ray Tune mainly helps you spend the same total compute more efficiently in wall-clock time via parallel trials + ASHA early-stopping, so higher iterations only makes sense when you actually have spare parallel capacity (and you should set gpu_per_trial to avoid oversubscribing). The behavior/args are documented in the Model.tune() reference and the Ray Tune integration guide.