Candidates Engine

[AdrianSosic]

Fixes #793

PRs merged

• Refactor Campaign #821
• Add CandidatesProtocol #800
• Refactor searchspace attributes #778
• Fix Serialization #833
• Recommendation Dataframes #832
• Deprecate SubspaceDiscrete.constraints #835
• Refactor recommendation mechanics #838
• Refactor candidates interface #840
• Wire CandidatesProtocol into SubspaceDiscrete #842
• Backend Settings #865
• Narwhalify DiscreteParameter.transform #862
• Narwhalify transformation execution chain #867
• Narwhalify recommendation layer #887

Upcoming PRs

• Narwhalify model layer #913
• Add sequence parameter #866
• Lazify get_candidates #915

To be decided

• Should the is_constrained property of the search space classes be dropped? Reasons:
  • SubspaceContinuous.is_constrained used to contain non-trivial logic but with the new design it's equivalent to a simple if self.constraints call
  • SubspaceDiscrete.is_constrained was very much meaningless since the constraints attribute itself was flawed (Deprecate SubspaceDiscrete.constraints #835). And with both the legacy and new design, it's not obvious what it actually means!
  • Having an is_constrained attribute only on some of the search space classes is asymmetric
• SubspaceDiscrete.batch_constraints vs DiscreteBatchConstraint name conflict. Options:
  • rename batch_constraints to something else, e.g. recommendation_constraints. However, batch_constraints is already a generic term. Perhaps we should rather ...
  • rename DiscreteBatchConstraint to something more specific. The latter is actually rather a name for an abstract base class if we decide to add more batch-level constraints, and does not convey anything about what it does. Options would be in the direction of DiscreteSharedValueConstraint
• Can we fully kick n_batches_done and n_fits_done?
• How to cleanly separate "filtering" and "batch constraints"? The eval_during_creation and eval_during_modeling are currently not mutually exclusive, so the semantics are not 100% clear, hence the assert statements in searchspace/discrete.py <-- clean up during constraint refactoring?
• Scaling approach: shall we use complete unfiltered space, filtered space or policy-generated subset? Decision depends heavily on what is cheap and possible in a large lazy space but also what makes conceptually sense (i.e. should adding a hypothetical candidate that is removed through a filter, e.g. by policy or active values, change the induced scaling?). Also impacts methods like comp_rep_bounds, which now behave differently compared to main (Refactor candidates interface #840 (comment)). At the same time, the discrete version of comp_rep_bounds may be dropped entirely since effectively unused at the moment (it's called in optimize_acqf_mixed but the discrete bounds should actually not matter since the candidates are explicitly piped in as separate argument --> needs investigation)
• Decorrelation: same problem as for scaling above
• Naming: EmptyCandidates vs NullCandidates
• encode vs transform vs computational representation terminology. Transform is currently used for continuous-valued objects (e.g. target transforms) but also for in the parameter/searchspace context. However, the latter also uses encoding (e.g. discrete parameters have specified encodings, we have a correspondingly named base class, ...). We somehow need to make this more precise. Perhaps: transform for conti mappings and encode for discrete mappings? Also covers things like the _encoding_table methods etc.
• move batch constraints to recommender?
• should we make any guarantees on (preserving) pandas indices? Mention in changelog/docs? If preserved --> tests?

TODOs

• Freeze SubspaceDiscrete and turn comp_rep into cached_property

Follow-up TODOs (after dev completion)

• Streamline recommendation call chain (i.e. avoid jumping back and forth between classes)