Ax

• Bump required Botorch version to v0.10.0
• New SLURM (via https://github.com/facebookincubator/submitit) tutorial notebook
• Miscellaneous big fixes and improvements

Compatibility

• Bump required BoTorch version to 0.9.5 (#2065).
• Unpin typeguard dependency (#2044).

New Features

• Allow batch trial to be constructed with a list of GeneratorRuns (#1995).
• Add label_dict to tile plots (#2007).
• Augment exp_to_df with a "reason" column, improve readability of "feasibility" column (#1973, #2047).
• Create output message comparing baseline to optimal result in report_utils.py (#1997, #1998, #2016, #2025, #2031, #2042, #2046, #2050).
• Allow custom search spaces in get_experiment_with_observations (#2027).
• Partial support for GenerationNodes in GenerationStrategy (#1985, #1986, #1991, #2002, #2003, #2018, #2019, #2024, #2033, #2034, #2045).

Bug Fixes

• Fix usage of batch shape for warp transform (#1994).
• Use default dtype in Experiment.clone_with (375bf471debbe9154ce6b1c6e302e8a115a24bb4).
• Change DerelativizeTransform to not use model predictions when use_raw_status_quo is True or when the status quo is infeasible (#2036).

Deprecations

• Rename Models.BOTORCH to Models.LEGACY_BOTORCH (#1981).

Other Changes

• Do not call dataset.X in input transform constructors (#1993).
• Move pending point utils to core Ax (#2006).
• Load Experiment without runners and metrics in the case where search space and optimization config are immutable (#1656).
• Workaround for tutorial visualizations not working in colab and remote setup (#2030).
• Add extract_pending_observations function that auto-deploys to the correct pending points function for the use case (#2039).
• Wait to re-poll if all results are MetricFetchE (#2055).
• Benchmarks:
  • Update BestPointMixin to support BatchTrial in benchmarks (#2014).
  • Adjust BenchmarkProblem to support multiple noise levels (#2049).
  • Make distribute_replications a required input argument (#2051).

• Bump required botorch version to 0.9.4
• Miscellaneous bug fixes and improvements

• Bump required botorch version to 0.9.2, fixing major bug in single-objective optimization with outcome constraints, see botorch release 0.9.2 for details

• Remove typeguard usage in trial attaching function and replace with manual runtime type checking (this was causing errors for some users in Google Colab notebooks)
• Miscellaneous bug fixes and improvements

• Bump required Botorch version to v0.8.5
• Miscellaneous big fixes and improvements

• Bump required Botorch version to v0.8.3
• Pin typeguard to version 2.13.3 while we investigate best course of action for dealing with backwards incompatible changes introduced in v3.0.0

• Bump required botorch version to 0.8.2
• Pinned sqlalchemy version to <2.0. We will update Ax to be compatible with the newly released sqlalchemy 2.0 in the near future
• Changes to Modular Botorch Model allow for heterogeneous modeling (i.e. many surrogates, one acquisition function). A tutorial jupyter notebook will be posted on ax.dev soon.
  • Added optional argument surrogate_specs to BoTorchModel: an Optional Mapping of names onto SurrogateSpecs, which specify how to initialize specific Surrogates to model specific outcomes. If None is provided a single Surrogate will be created and set up automatically based on the data provided.
  • Deprecated ListSurrogate (subsumed functionality into Surrogate)
• Removed Models.MOO_MODULAR (Models.BOTORCH_MODULAR supports multi-objective setups)
• Support partial objective thresholds
• Miscellaneous testing speedups
• Miscellaneous bug fixes

• Bump required botorch version to 0.8.0
• Enable relative outcome constraints
• Misc bugfixes and improvements
  • Bugfix in BestPointMixin.get_trace
  • Avoid unnecessary model re-fitting in some cases
  • Allow inferred noise in benchmarking via infer_noise flag

• Revamped Metric API
  • Metrics now return Result type for data fetching methods. This allows metrics to encode exceptions encountered during fetching into their returned value so Ax can defer their raising.
    • Call experiment.fetch_data_results to access these Results (organized by trial index and metric name) on the Experiment level.
      • experiment.fetch_data still returns Data directly by unwrapping the results under the hood for backwards compatibility considerations.
    • To update your existing custom Metrics you must wrap the return value of fetch_trial_data with Ok.
      • See the Scheduler tutorial for an example
• Improvements to Scheduler Exception handling via Results
  • If an Exception is encountered by the Scheduler during data fetching, either mark the trial as FAILED or continue the optimization (while alerting the user) depending on the failed metric’s role in the optimization. See scheduler.py for exact rules.
• SAASBO memory utilization improvements
• Miscellaneous bugfixes

Bugfix for for compatibility with latests pandas release. Also upgrade to BoTorch v0.7.2

Bug fixes & improvements. Now using BoTorch v0.7.2.

Bugfixes and incremental improvements. Now using botorch v0.6.6

Bugfixes and incremental improvements

• Bugfixes
• Changes to saving custom Metrics and Runners (see Developer API Tutorial for details)

• Bugfixes
• SAASBO fixes
• Updates to Scheduler (base class now works standalone, with polling functionality delegated to runner)
• Early-stopping support in Service API

Bugfixes + support for Pareto frontier extraction in Ax Service API

What’s new?

1. Support Multi-Objective Optimizations for the Service API.
2. Deprecation of SimpleExperiment
3. Several minor tutorial and documentation corrections.
4. Documentation is no longer released with errors in tutorials.

Bugfixes

• IndexError when only using a subset of fields in raw data (https://github.com/facebook/Ax/issues/652)

What’s new?

1. Modular BotAx + tutorial: new flexible setup for combining BoTorch subcomponents into a Model in Ax,
2. Scheduler + tutorial: new standalone configurable manager abstraction to run asynchronous closed-loop experiments with Ax (with or without human-in-the-loop),
3. GenerationStrategy tutorial: walkthrough for optimization algorithms specification in in Ax,
4. Bayesian optimization with categorical kernels exposed in Ax and auto-selected in Service and Loop APIs (via choose_generation_strategy). This eliminates the need to opt for Sobol quasi-random search when a given search space contains predominantly choice (a.k.a. categorical or discrete) parameters,
5. Scalarized outcome constraint support (feature request: https://github.com/facebook/Ax/issues/472),
6. Extended exp_to_df utility to conveniently display an Ax Experiment as a dataframe of trials and arms.

Bugfixes

• Abandoned arms and trials exclusion from modeling data (https://github.com/facebook/Ax/issues/506),
• Changing a range parameter to fixed (https://github.com/facebook/Ax/issues/383).

Compatibility: Requires botorch == 0.4.0

Major changes

• Switching from Travis CI to Github Actions for continuous integration and deployment
• New tutorial for using BoTorchModel, which makes it easier to create custom BoTorch models in Ax (https://github.com/facebook/Ax/commit/88c4a3f572455254f11470fcf977478c9cbfc229)
• Fixed the Raytune and MultiTask tutorials
• Multiple bug fixes (e.g. https://github.com/facebook/Ax/issues/501, https://github.com/facebook/Ax/issues/471)