Alternate optimizers

[MTakahashi-KWH]

Keeping in draft while the rest of the optimizers provided by scipy optimize are filled in on the SCP enum (should be pretty procedural with no changes needed to testing except regarding global optimizers)

---

Request introduces a new Scipy-based gradient optimization interface to the Blop library and provides some documentation and usability improvements for gradient-based optimization workflows. The main changes include the addition of a user-friendly Scipy agent interface and standard independent ScipyOptimizer (with/session interface and standard object usage), new documentation for Scipy optimization, and enhancements to DOF support for Scipy optimizers.

New Scipy optimization interface and documentation:

• Rerigged simple-experiment.md as a system_spanning tutorial (gradient-optimization.md) demonstrating how to use Blop with Scipy optimizers, including setup, defining DOFs/objectives, evaluation functions, running optimizations, and visualizing results.
• Introduced a new Scipy class in src/blop/gradient/Scipy.py that provides a convenient, Ax-like agent interface (Scipy.Agent(...)) for running gradient-based optimizations with Scipy, supporting both single and multipoint sampling, callback management, and integration with Bluesky.
  • Domain native configuration interface (ScipyCFG(...) and Scipy(ScipyCFG)) also provided for deeper customization of scipy optimizers

Enhancements to DOF/parameter handling:

• Added a to_scipy_bounds method to the RangeDOF class, enabling easy conversion of DOFs to Scipy's Bounds object for compatibility with Scipy optimizers. (may be dropped due to 2 scipy optimizers unable to support bounds??)
• Imported Scipy's Bounds in src/blop/ax/dof.py to support the new conversion method.

Targets:

#282 Add gradient based methods to default package capabilities

[MTakahashi-KWH]

This is now a mostly complete implementation. There are some unexposed optimizers and scipy options that I believe should be pushed to a future update of this module.

[thopkins32]

Fairly complex implementation so I will have to review once more after discussion. Are there practical ways to simplify a lot of this?

The major complexity stems from the required cooperative thread + callback mechanism and the support for various algorithms from scipy.optimize. That + some state tracking should be all we need to get an optimizer working with Blop's protocols.

If we defer to the user to give us the scipy.optimize method (w/ *args, **kwargs) in the scipy Agent/Optimizer, then all we have to do is state tracking + cooperative thread management.

Our "cost function", in scipy terms, is always to simply wait (possibly forever) until the next call to ingest(), nothing more, nothing less.

[thopkins32]

Enums must be all caps, surprised the ruff linter didn't catch this.

[thopkins32]

See my comment on API design for this. Can users pass in the scipy.optimize method they want instead of us maintaining a "supported algorithm" list?

[MTakahashi-KWH]

This is a debate I've have with development as Scipy is notorious for having an inconsistent API. I decided to 'flatten' the call so that a standard parameter/call format fits them all, and needed enum as a gatekeeper for as I expand the implementation support. Adding the ability to pass an optimizer would would need a standard enough parameter interface Scipy.opt doesnt have (see some algorithms also requiring a jacobian or an analytical gradient closure too)

In a sense my implementations primary development is to invert the prompting loop of a cost closure calling optimization function and not necessarily scipy specific. I can definitely make a generic loop inversion implementation and regularize scipy's calls in another file to work with this and we'd just pass one of those as something like "loop_invert(scipy_norm.BFGS,*params)". Another optimization system that accepts a cost function closure could run the same way. Though ultimately that's a consequence of the implementation, not the objective. The design started from the assumption that SciPy is the optimization backend to enable gradient optimization, not from a desire to abstract over arbitrary optimizers. I see no significant evidence that another optimization library operates in a similar way to enable this.

[thopkins32]

Yeah I realize this is a bit annoying. Most of these methods take a callable function as the first argument though which is something we can standardize on. In fact, all of the methods under the "Optimization" sub-header take func as the first argument.

This would enable the use of:

• minimize_scalar
• minimize with any choice of method
• basinhopping
• brute
• differential_evolution
• shgo
• dual_annealing
• direct

This already covers so much that I think it's fine to start here. If there is demand for anything beyond this, then we can revisit.

[thopkins32]

I am confused by this. Why is it necessary?

[MTakahashi-KWH]

This is from the fact that we don't have a stopping condition and agent.optimize() runs for a fixed number of iters. If scipy converges to a solution, it will stop creating sample points. I made the decision that if it converges in the middle of an optimization, a user would be more happy for it to run the iterations to completion without erroring out run engine. But I still give the ability to do so anyways to cut useless operation.

[MTakahashi-KWH]

added refactoring branch scipy_barebones which separates the parameter normalization of inner/closed optimizer call (InnerOptimizer) from the inversion setup (OuterOptimizer). Really need a better naming convention for this structure

of interest to perhaps:
@thopkins32
@josephhanrahan