vizier

What's Changed

• Fix bug in predict method of gp_bandit.py (not able to pass rng) by @copybara-service in https://github.com/google/vizier/pull/1146
• Fix xlabel and debug state analyzer histogram plot. by @copybara-service in https://github.com/google/vizier/pull/1147
• Make the client test fixture class generic so subclasses can write platform-specific tests. by @copybara-service in https://github.com/google/vizier/pull/1148
• Add return type of fig for plotting. by @copybara-service in https://github.com/google/vizier/pull/1149
• In the Halton quasi-random sampler, we were requesting D samples when we only by @copybara-service in https://github.com/google/vizier/pull/1151
• Relax numerical tolerance in preparation for xla:cpu update by @copybara-service in https://github.com/google/vizier/pull/1152
• 1. Add AcquisitionOverScalarized (Scalarize distributions first to obtain scalar, then get UCB) by @copybara-service in https://github.com/google/vizier/pull/1153
• 1. Add AcquisitionOverScalarized (Scalarize distributions first to obtain scalar, then get UCB) by @copybara-service in https://github.com/google/vizier/pull/1154

Full Changelog: https://github.com/google/vizier/compare/v0.1.17...v0.1.18

What's Changed

• Add decorator @seed_with_default by @copybara-service in https://github.com/google/vizier/pull/1086
• Fix bug on allowed version name by @copybara-service in https://github.com/google/vizier/pull/1092
• Use TEST_PYPI secrets instead by @copybara-service in https://github.com/google/vizier/pull/1093
• Upgrade jax versions to avoid DeviceArray import issues. by @copybara-service in https://github.com/google/vizier/pull/1094
• Support for infeasible trials in ensemble_designer by @copybara-service in https://github.com/google/vizier/pull/1090
• ...added validation code to satisfy PyType Checks... by @copybara-service in https://github.com/google/vizier/pull/1096
• Adds the pe_overwrite_probability parameter to the UCBPEConfig by @copybara-service in https://github.com/google/vizier/pull/1091
• Updates the default value for pe_overwrite_probability by @copybara-service in https://github.com/google/vizier/pull/1101
• Add HashingInfeasibleExperimenter and ParameterRegionInfeasibleExperimenter to simulate infeasibility behaviors. by @copybara-service in https://github.com/google/vizier/pull/1104
• Add to public import infeasible experimenters by @copybara-service in https://github.com/google/vizier/pull/1105
• Parameterized the penalty factor and rename it. by @copybara-service in https://github.com/google/vizier/pull/1103
• Default trial seeding, without using scalers. by @copybara-service in https://github.com/google/vizier/pull/1088
• Implements a variant of GP_UCB_PE that performs exploration via set-optimization by @copybara-service in https://github.com/google/vizier/pull/1110
• Supports multi-batch dimensions in the trust region acquisition function. by @copybara-service in https://github.com/google/vizier/pull/1114
• Appends the first suggestion to active trials before generating the remaining suggestions in a batch by @copybara-service in https://github.com/google/vizier/pull/1116
• Passes prior features to set-acquisition optimization by @copybara-service in https://github.com/google/vizier/pull/1117
• Add a wrapper to allow fairer comparisons between Vizier and other raytune algorithms. by @copybara-service in https://github.com/google/vizier/pull/1118
• Small update by @copybara-service in https://github.com/google/vizier/pull/1106
• Add simple4d convergence test for gp bandit. by @copybara-service in https://github.com/google/vizier/pull/958
• Add BenchmarkRecord summarization tools. by @copybara-service in https://github.com/google/vizier/pull/1120
• Refactor experimenter factories by @copybara-service in https://github.com/google/vizier/pull/1121
• Add xlabel to PlotElement. by @copybara-service in https://github.com/google/vizier/pull/1123
• Deprecate predictive_fns.py from Vizier JAX. by @copybara-service in https://github.com/google/vizier/pull/1126
• Fix accidental bug in MultiObjectiveNumpyExperimenter by @copybara-service in https://github.com/google/vizier/pull/1129
• Aesthetic + small bug fixes to Vizier service by @copybara-service in https://github.com/google/vizier/pull/1130
• Revert Twine=5.0.0 to unblock deployment failures: https://github.com/pypa/twine/issues/1125 by @copybara-service in https://github.com/google/vizier/pull/1131
• Enables GP_UCB_PE to explore more aggressively in high-noise situations. by @copybara-service in https://github.com/google/vizier/pull/1133
• Change k = num_objectives - 1 by @copybara-service in https://github.com/google/vizier/pull/1134
• Use reference point and multiple scalarizations in ScalarizedAcquisition. by @copybara-service in https://github.com/google/vizier/pull/1135
• Vectorize the scalarization method instead of producing multiple class instances. This should run faster + reduce the jitted graph size (prevent RAM blowups) by @copybara-service in https://github.com/google/vizier/pull/1136
• Use 1000 scalarizations as default. Vectorization made it much faster to pass unit tests by @copybara-service in https://github.com/google/vizier/pull/1138
• Add Sample acquisition function to GP bandit. by @copybara-service in https://github.com/google/vizier/pull/1141
• Linear kernel refactor: Merge VizierGaussianProcess with VizierLinearGaussianProcess by @copybara-service in https://github.com/google/vizier/pull/1142
• Add max_scalarized to ScalarizedAcquisition. by @copybara-service in https://github.com/google/vizier/pull/1143
• Refactor multiobjective setup by @copybara-service in https://github.com/google/vizier/pull/1144
• Prepare for 0.1.7 release by cleaning up multiobjective. by @copybara-service in https://github.com/google/vizier/pull/1145

Full Changelog: https://github.com/google/vizier/compare/v0.1.16...v0.1.17

What's Changed

• Reduce duplicate code by @copybara-service in https://github.com/google/vizier/pull/1046
• Fix endpoint issues in pyglove colab. Should resolve https://github.com/google/vizier/issues/1044 by @copybara-service in https://github.com/google/vizier/pull/1047
• Fix link by @copybara-service in https://github.com/google/vizier/pull/1050
• Updates acquisition optimizer parameters for GP_UCB_PE by @copybara-service in https://github.com/google/vizier/pull/1051
• Change subplot title depending on if experimenter key is present. by @copybara-service in https://github.com/google/vizier/pull/1052
• Adds an OSS GP_UCB_PE test by @copybara-service in https://github.com/google/vizier/pull/1061
• Incorporate infeasible trials into Eagle designer by @copybara-service in https://github.com/google/vizier/pull/1057
• Add more details on adaptive ensembling. by @copybara-service in https://github.com/google/vizier/pull/1066
• Internal clean up. by @copybara-service in https://github.com/google/vizier/pull/1064
• Adds shift to InfeasibleWarperComponent and turns it on for the default output warper pipeline by @copybara-service in https://github.com/google/vizier/pull/1067
• Add Branin and Hartmann to public import by @copybara-service in https://github.com/google/vizier/pull/1068
• Update linear output warper to work nicely with jit caching. by @copybara-service in https://github.com/google/vizier/pull/1070
• Update logic for ensemble designer metadata failure into a log warning (in most cases). by @copybara-service in https://github.com/google/vizier/pull/1072
• Update scheduled params API by @copybara-service in https://github.com/google/vizier/pull/1073
• Stores per-suggestion latency in GP_UCB_PE suggestion timing metadata by @copybara-service in https://github.com/google/vizier/pull/1075
• Speed up pareto_rank computation. by @copybara-service in https://github.com/google/vizier/pull/1078
• Clarify and consolidate convergence curve scores by @copybara-service in https://github.com/google/vizier/pull/1076
• Serialize the scheduled designer. by @copybara-service in https://github.com/google/vizier/pull/1074
• Update benchmarking colab + reduce unnecessary imports by @copybara-service in https://github.com/google/vizier/pull/1081
• Simplify Ray benchmarks RTD by @copybara-service in https://github.com/google/vizier/pull/1082
• Update to 0.1.16 by @copybara-service in https://github.com/google/vizier/pull/1084

Full Changelog: https://github.com/google/vizier/compare/v0.1.15...v0.1.16

What's Changed

• Add GP_UCB_PE and make it the new default. by @copybara-service in https://github.com/google/vizier/pull/1042

Full Changelog: https://github.com/google/vizier/compare/v0.1.14...v0.1.15

What's Changed

• Add a json encoder for metadata by @copybara-service in https://github.com/google/vizier/pull/1009
• Add failing designer for testing. by @copybara-service in https://github.com/google/vizier/pull/1010
• Allow for linear kernel for ARD in Python GP. by @copybara-service in https://github.com/google/vizier/pull/1012
• Fix early stopping colab by @copybara-service in https://github.com/google/vizier/pull/1014
• Modernize @jaxtyped code by @copybara-service in https://github.com/google/vizier/pull/1015
• Support reverse log in keras embedder and small edge case fix in core converter. by @copybara-service in https://github.com/google/vizier/pull/1017
• Support SimpleKd experimenter returning relative objective value. by @copybara-service in https://github.com/google/vizier/pull/1011
• Propagate selected keys from global namespace to metadata namespace. by @copybara-service in https://github.com/google/vizier/pull/1023
• Simplify some SQL datastore code by @copybara-service in https://github.com/google/vizier/pull/1022
• gp_bandit designer rejects empty search spaces. by @copybara-service in https://github.com/google/vizier/pull/1024
• No need for union since vz.Trial is already a subclass by @copybara-service in https://github.com/google/vizier/pull/1025
• Debug shifting experimenter due to OOB values (and revert to previous code). by @copybara-service in https://github.com/google/vizier/pull/1019
• Apply suggestion from ConnorBaker in https://github.com/google/vizier/pull/1029 by @copybara-service in https://github.com/google/vizier/pull/1033
• Remove Python 3.7 support and fix ugly formatting in pyglove core by @copybara-service in https://github.com/google/vizier/pull/1035
• Allow for mutation rate schedules for NSGA by @copybara-service in https://github.com/google/vizier/pull/1034
• Update benchmark result structure to incorporate spec gen name. by @copybara-service in https://github.com/google/vizier/pull/1020
• Adds a new error type (not yet fully supported) that allows Pythia algorithms by @copybara-service in https://github.com/google/vizier/pull/1038
• Some support for PythiaFallbackError. by @copybara-service in https://github.com/google/vizier/pull/1039
• Adds benchmarks testing. by @copybara-service in https://github.com/google/vizier/pull/1040
• Add NormalizedSimpleRegert and WinRateSimpleRegret convergence curve comparators. by @copybara-service in https://github.com/google/vizier/pull/1041
• Creating scheduled GP-Bandit designer. by @copybara-service in https://github.com/google/vizier/pull/1043

Full Changelog: https://github.com/google/vizier/compare/v0.1.13...v0.1.14

What's Changed

• Remove logging errors when the convertor rounds up values, which is not an uncommon occurrence (e.g. RandomDesigner, EagleStrategyDesigner). by @copybara-service in https://github.com/google/vizier/pull/950
• Input validation for add_categorical_param. by @copybara-service in https://github.com/google/vizier/pull/952
• Fix the type error. nn.module.get_variable returns a regular dict now. by @copybara-service in https://github.com/google/vizier/pull/957
• Fixes Eagle optimizer trials padding by @copybara-service in https://github.com/google/vizier/pull/960
• ParameterEntropyHparamScorerFactory for scoring exploration behavior by @copybara-service in https://github.com/google/vizier/pull/970
• Add prior study setters into benchmark runners. by @copybara-service in https://github.com/google/vizier/pull/984
• Update documentation: Users are now allowed to write to namespaces other than by @copybara-service in https://github.com/google/vizier/pull/988
• Introduce stateful curve converters. by @copybara-service in https://github.com/google/vizier/pull/987
• Add support for conditionals in TrialToArrayConverter. by @copybara-service in https://github.com/google/vizier/pull/1001
• Minor edits by @copybara-service in https://github.com/google/vizier/pull/1000

Full Changelog: https://github.com/google/vizier/compare/v0.1.12...v0.1.13

What's Changed

• Small aesthetic update to Ray benchmark RTD by @copybara-service in https://github.com/google/vizier/pull/914
• Add a lot of tracing trackers. by @copybara-service in https://github.com/google/vizier/pull/913
• Fix broken pytest_ray installation by @copybara-service in https://github.com/google/vizier/pull/918
• Add multi-objective acquisitions and integrate them into GPBandit. by @copybara-service in https://github.com/google/vizier/pull/909
• Add profiling around seed trial generation to help debug slow runs. by @copybara-service in https://github.com/google/vizier/pull/925
• Meta learning designer: 1) multi-metric trial support, 2) updated default schedules, 3) additional testing and documentation. by @copybara-service in https://github.com/google/vizier/pull/933
• Add counter metrics with metadata. by @copybara-service in https://github.com/google/vizier/pull/931
• Use QuasiRandomDesigner in EagleStrategy. by @copybara-service in https://github.com/google/vizier/pull/943
• Clean up & bugfix for random rotation matrix generation. by @copybara-service in https://github.com/google/vizier/pull/941
• Remove redundant logging from Eagle designer for expediting benchmarking jobs / convergence tests. by @copybara-service in https://github.com/google/vizier/pull/946

Full Changelog: https://github.com/google/vizier/compare/v0.1.11...v0.1.12

What's Changed

• Expose PolicyFactory into Pythia import. by @copybara-service in https://github.com/google/vizier/pull/905
• General Meta-Learning Framework by @copybara-service in https://github.com/google/vizier/pull/877
• Debug ensemble_designer to handle with Active Trials. by @copybara-service in https://github.com/google/vizier/pull/910

Full Changelog: https://github.com/google/vizier/compare/v0.1.10...v0.1.11

What's Changed

• Clarify that in multi-worker situations, workers are expected to provide client_id in the suggest() call, and not during client construction. by @copybara-service in https://github.com/google/vizier/pull/875
• Expose DesignerPolicy in algorithms init file by @copybara-service in https://github.com/google/vizier/pull/894
• Update Vizier version to 0.1.10 by @copybara-service in https://github.com/google/vizier/pull/896

Full Changelog: https://github.com/google/vizier/compare/v0.1.9...v0.1.10

What's Changed

• Add PlotElements and BenchmarkRecord for easy plotting. by @copybara-service in https://github.com/google/vizier/pull/878

Full Changelog: https://github.com/google/vizier/compare/v0.1.8...v0.1.9

What's Changed

• 'all' pip README update by @copybara-service in https://github.com/google/vizier/pull/851

Full Changelog: https://github.com/google/vizier/compare/v0.1.7...v0.1.8

What's Changed

• Predict Kumamon individual metrics. by @copybara-service in https://github.com/google/vizier/pull/739
• Add GP Bandit tests for VizierSearcher by @copybara-service in https://github.com/google/vizier/pull/750
• Make the number of unpadded features a dynamic arg for acquisition function optimization so that padding/masking the feature dimension avoids retracing. by @copybara-service in https://github.com/google/vizier/pull/757
• Add a Converter from trials to ContinuousAndCategoricalArray. by @copybara-service in https://github.com/google/vizier/pull/754
• Minor cleanups by @copybara-service in https://github.com/google/vizier/pull/788
• Use Categorical kernel in GP bandit. by @copybara-service in https://github.com/google/vizier/pull/784
• Convert vectorized optimizers to operate on continuous and categorical data. by @copybara-service in https://github.com/google/vizier/pull/793
• Follow up to the VectorizedBase refactor with a couple fixes: by @copybara-service in https://github.com/google/vizier/pull/797
• Add arch gym to highlights by @copybara-service in https://github.com/google/vizier/pull/804
• Add the wide LCB and delayed trust region application options to the acquisition-based trust region. by @copybara-service in https://github.com/google/vizier/pull/805
• Silence some pytype errors. by @copybara-service in https://github.com/google/vizier/pull/820
• Minor updates to pythia docs, string reps, and TODOs by @copybara-service in https://github.com/google/vizier/pull/821
• Internal change. by @copybara-service in https://github.com/google/vizier/pull/829
• Advance attrs version to 23.1.0 by @copybara-service in https://github.com/google/vizier/pull/830
• Add batching/ensembling support for mean_fn. by @copybara-service in https://github.com/google/vizier/pull/834

Full Changelog: https://github.com/google/vizier/compare/v0.1.6...v0.1.7

What's Changed

• Fix qUCB test following a change to TFP. Rename qUCB parameter "coefficient" for consistency with UCB. by @copybara-service in https://github.com/google/vizier/pull/691
• Bug fix/add missing test cases for Optax ARD optimizer. by @copybara-service in https://github.com/google/vizier/pull/692
• Add option to pad number of observations and dimensions of inputs. by @copybara-service in https://github.com/google/vizier/pull/627
• Pyvizier needs to include one more definition. by @copybara-service in https://github.com/google/vizier/pull/693
• Update Adaptive Ensembling to use observation probs. by @copybara-service in https://github.com/google/vizier/pull/695
• Clarify jit_loop arg in vectorized_base.py by @copybara-service in https://github.com/google/vizier/pull/705
• Fix the flag error in runtime context. https://github.com/google/vizier/issues/698 by @copybara-service in https://github.com/google/vizier/pull/700
• Update Basic Vizier documentation by @copybara-service in https://github.com/google/vizier/pull/716
• Update all Jax requirements to avoid breaking Github workflows by @copybara-service in https://github.com/google/vizier/pull/722
• Disable benchmark tests for now due to Dopamine failures. by @copybara-service in https://github.com/google/vizier/pull/730
• Remove 3.x from github workflow by @copybara-service in https://github.com/google/vizier/pull/731

Full Changelog: https://github.com/google/vizier/compare/v0.1.5...v0.1.6

What's Changed

• Create input warping based on Kumaraswamy CDF. by @copybara-service in https://github.com/google/vizier/pull/562
• Fix Numpy dtype error in acquisitions_test. by @copybara-service in https://github.com/google/vizier/pull/605
• Add a test to the RandomDesigner to confirm that it respects LOG scaling. by @copybara-service in https://github.com/google/vizier/pull/596
• Fix Eagle slowness for large studies by @copybara-service in https://github.com/google/vizier/pull/609
• Add typing to output warping by @copybara-service in https://github.com/google/vizier/pull/613
• Use time as seed by @copybara-service in https://github.com/google/vizier/pull/614
• Fix pytype failures related to teaching pytype about NumPy scalar types. by @copybara-service in https://github.com/google/vizier/pull/604
• Add RayTune SearchSpace and Experimenter Converters and test on mock study. by @copybara-service in https://github.com/google/vizier/pull/615
• Update GoogleDesignerFactory to use a designer factory. by @copybara-service in https://github.com/google/vizier/pull/611
• Remove GP hparam ensemble since ensemble members are usually the same with the LBFGSB optimizer, and to reduce latency associated with jax.vmap with the forthcoming JAX Eagle optimizer. Change best_n=1 behavior to squeeze out the singleton dimension of the GP hparams and not use vmap, since unnecessary vmap is a speed footgun. by @copybara-service in https://github.com/google/vizier/pull/618
• Normalize loss value by the number of observations only when using SGD-based GP hparam optimizers (so a consistent learning rate can be used regardless of the number of observations). Previously, this was baked into the loss function definition; now that we're using L-BFGS-B by default, this is unnecessary and possibly confusing. by @copybara-service in https://github.com/google/vizier/pull/602
• Update Pytypes after recent changes to typing in JAX/Numpy, and following our decision to use more specific types (instead of chex.Array) where possible. by @copybara-service in https://github.com/google/vizier/pull/624
• Fix a bug where Constraint objects returned by get_constraints did not account for mean_fn parameters. by @copybara-service in https://github.com/google/vizier/pull/633
• Added utils/performance_utils.py with a decorator that tracks function runtime duration. by @copybara-service in https://github.com/google/vizier/pull/638
• Make 'num_metric' the last dimension in LinearOutputWarper by @copybara-service in https://github.com/google/vizier/pull/648
• Kumamon: apply linear output warping and compute acquisition for separable kernel using sampling. by @copybara-service in https://github.com/google/vizier/pull/649
• Add more logs to DesignerPolicy to better track performance. by @copybara-service in https://github.com/google/vizier/pull/653
• Update mapping logic for Raytune. by @copybara-service in https://github.com/google/vizier/pull/657
• Add L-BFGS-B Acquisition function optimizer. by @copybara-service in https://github.com/google/vizier/pull/658
• Move NestedDictRAMDatastore to a new file and move metadata operations into metadata_util.py. by @copybara-service in https://github.com/google/vizier/pull/673
• Update Predictor pydoc. by @copybara-service in https://github.com/google/vizier/pull/676
• Update predict RTD colab. by @copybara-service in https://github.com/google/vizier/pull/681
• Fixes Trial default creation_time by @copybara-service in https://github.com/google/vizier/pull/684

Full Changelog: https://github.com/google/vizier/compare/v0.1.4...v0.1.5

What's Changed

• More highlights by @copybara-service in https://github.com/google/vizier/pull/543
• Small Doc Fix by @copybara-service in https://github.com/google/vizier/pull/547
• Validate parameters in quasi-random and be less confusing about "NumpyArraySpec.DISCRETE" vs Vizier Discrete/Categorical. by @copybara-service in https://github.com/google/vizier/pull/544
• Scalarizing Designer for MOO. by @copybara-service in https://github.com/google/vizier/pull/540
• Refactor and split docs into (Designers: Writing Algorithms) and (Pythia Policies and Hosting Designers). by @copybara-service in https://github.com/google/vizier/pull/551
• Add supported algorithms RTD by @copybara-service in https://github.com/google/vizier/pull/555
• Simplify User API description. by @copybara-service in https://github.com/google/vizier/pull/557
• Refactor Benchmarks to interact with only Policies which makes for automatic data sync. All Designer specific logic for in-ram efficiency is siloed in DesignerPolicy. by @copybara-service in https://github.com/google/vizier/pull/554
• Need jax installation to run GP-Bandit in colab by @copybara-service in https://github.com/google/vizier/pull/560
• Multi-task GP with non-linearities. by @copybara-service in https://github.com/google/vizier/pull/517
• Additional shape check for the features and labels for DefaultTrialConverter. by @copybara-service in https://github.com/google/vizier/pull/559
• Expose some experimenters in benchmark init by @copybara-service in https://github.com/google/vizier/pull/567
• Move to a dedicated experimenters init. Also disable immutabledict import in SparseExperimenter. by @copybara-service in https://github.com/google/vizier/pull/570

Full Changelog: https://github.com/google/vizier/compare/v0.1.3...v0.1.4

What's Changed

• Support generic search spaces in Eagle Designer using ProblemAndTrialsScaler by @copybara-service in https://github.com/google/vizier/pull/504
• Clarify the usage of ns() and abs_ns() for metadata objects. by @copybara-service in https://github.com/google/vizier/pull/538
• Add hypervolume scalarization as Acquisition Function. by @copybara-service in https://github.com/google/vizier/pull/525
• Fix to Quasi-Random Designer in https://github.com/google/vizier/pull/542

Full Changelog: https://github.com/google/vizier/compare/v0.1.2...v0.1.3

What's Changed

• Add immutabledict import by @copybara-service in https://github.com/google/vizier/pull/478
• Remove pointless ensure-compile-time decorator. by @copybara-service in https://github.com/google/vizier/pull/486
• Change how Cholesky factors are cached when computing the posterior predictive to accommodate an upcoming change to Jax. by @copybara-service in https://github.com/google/vizier/pull/498
• Update RTD Colab following a change to StochasticProcessModel.predict. by @copybara-service in https://github.com/google/vizier/pull/507
• More media by @copybara-service in https://github.com/google/vizier/pull/509
• Add a test for a Stochastic Process Model that builds a multitask GP. by @copybara-service in https://github.com/google/vizier/pull/503
• Correct usage example for completing infeasible trials. by @copybara-service in https://github.com/google/vizier/pull/513
• More media + reorganizing by importance by @copybara-service in https://github.com/google/vizier/pull/518
• Clarify docstring for pyvizier.Namespace class. by @copybara-service in https://github.com/google/vizier/pull/524
• Internal Change. by @copybara-service in https://github.com/google/vizier/pull/530

Full Changelog: https://github.com/google/vizier/compare/v0.1.1...v0.1.2

What's Changed

• Use batch_size=1 since we're now using VizierGPBandit by @copybara-service in https://github.com/google/vizier/pull/470
• Update to 0.1.1 by @copybara-service in https://github.com/google/vizier/pull/473
• Intensify performance_test while removing redundant vizier_server_test by @copybara-service in https://github.com/google/vizier/pull/474
• Add "getting started" code snippet to README. by @copybara-service in https://github.com/google/vizier/pull/475

Full Changelog: https://github.com/google/vizier/compare/v0.1.0...v0.1.1

Official Release for AI Blog post.

Improved PyGlove integration