Neuraxle

General improvements and bugfixes:

• Update FlattenForEach that wasn't up to date.
• Use _ids, data_inputs, and expected_outputs more often in DataContainer instead of shorthand ids, di, and eo, which were auto-filling values too often when looping over things in flow classes and output handlers.
• Changed InputAndOutputTransformerMixin to IdsAndInputAndOutputTransformerMixin and derived classes to also process IDs more often in triplets rather than duo tuples of di and eo.
• Fixed bugs in DataContainer.
• Easy to read string representation of the DataContainer (DACT) is now possible for easier print debugging at a glimpse.
• Repair some skipped tests.
• Cleaned some docstrings.
• Add __str__ and __repr__ functionalities to context to show its services and parents in detail upon printing.
• _TruncableMixin that is common to _TruncableSteps and _TruncableService.
• Introducing _TruncableServiceWithBodyMixin for .body and .joiner that is easier. Also fix FlattenForEach.
• Added different copy constructors to services depending on the AutoML train/val phase.
• Add the .mutate(...) function again in the services and steps.
• Add the .will_mutate_to(...) function again in the services and steps.
• Rename copy() to _copy() in the services and ExecutionContext to bypass the fact that the copy method was already defined in some python core data structures. This renaming avoids conflicting these functionalities of the core python libs and of Neuraxle when defining services that inherits from core data structures at the same time.
• Improvements to the _repr to make step strings less bloated when debugging: removed steps names and steps hyperparams when names are redundant with class names and hyperparams empty. Also sometimes the str will be a compact one-liner when the children of a truncable step are of length 1.

New major version number since new changes added in 0.7.1 and 0.7.2 and the following changes of 0.8.0 that makes debugging and usage of parallelism much easier:

• Lots of upgrades to the parallelization module of Neuraxle, which is now much more safer and works with logging to be able to debug exceptions and bubble up exceptions properly in unit tests as well. This quality of life improvement saves lots of debugging time even in the short term.
• Timeouts on tests to avoid deadlocks to hang in unit tester.
• Doing pickling checks on the parallelized services for them to avoid deadlocking the multithreading queues by having picklables services.
• It also repairs several bugs that impacted the parallelism that were sometimes causing deadlocks. Only a few race conditions seems to remain.
• Moving some modules around and removing some dependencies between modules for lighter workflow and clearer file names relatively to contained classes.

Minor release to add AutoML Report classes used to generate statistics:

• A new file called reporting.py in the AutoML module allows generating statistics on optimization rounds and other related objects.
• Such a report contains a dataclass of the same subclass-level of itself, so as to be able to dig into the dataclass so as to observe it, such as to generate statistics and query its information.
• Dataclasses represent the results of an AutoML optimization round, even multiple rounds.
• These AutoML reports are used to get information from the nested dataclasses, such as to create visuals.
• Just pass the dataclass to the reporting class, and do function calls.
• Example usage: BaseReport.from_dc(some_auto_ml_dataclass)
• Then call the methods for the statistics you want to compute for reporting.

Minor version release that allows for usage of SQLAlchemy ORM for hyperparameter repositories:

• Recursive tree table that joins on itself to load the vanilla data classes in depth.
• To represent the nodes, SQL polymorphism is used by joining on other tables.
• Abstract databases can use many technologies (e.g.: open to be implemented in SQLite, PostgreSQL, MySQL, and more).
• Some utility functions were added to dataclasses, such as ".tree()".

Major changes to the AutoML module are done in this version to improve its capabilities considerably:

• Feature: 3rd Version for AutoML module.
• Feature: ability to print epochs "i/n", and also the iter "j/m" of batches inside an epoch
• Fix: ValidationSplitWrapper should use the same code as ValidationSplitter.
• Fix: Cleanup all of the scipy distributions that are already available within the regular distribution module
• Feature: Ctor of Hp Space should validate that each object is of type Distribution.
• Feature: ExecutionContext could inherit from some steps mixins like a TruncableStep, and ExecutionContext Services could inherit from some mixins as well.
• Feature: Typing in meta steps ! (wrappers' subtypes) - like C++ data structures' content templates
• Feature: Improving Context to handle Assertions and Logging differently depending on the context.
• Fix: If using savers in ParallelQueuedPipeline
• Fix: Have the hyperparams setters (and space setters) of the TruncableSteps and MetaStepMixin throw errors when the name of the substep doesn't exist
• Feature: DeprecatedMixin
• Feature: Create _HasAssertsMixin to do things like self._assert(1 == 1, error_msg, context, level=None).
• Feature: Ditch IDs in the data container, and ditch re-hashing mechanisms
• Feature: Trainer class should be passed as an argument to AutoML
• Feature: Add BaseStep.config, BaseStep.get_config() and BaseStep.set_config()
• Fix: ForceHandlers things require overriding fit_transform
• Fix: higher_score_is_better=False while it should be true
• Fix: TPE LogNormal distributions

• Make Neuraxle work under Windows. May error sometimes with the file lock issues at deletions / teardowns, but that is a detail and things are mostly working.
• Add python 3.9 support.
• Removed support for pickle checkpoints and kept only joblib checkpoints to simplify maintenance.
• Add vscode to gitignore.
• Little debugging and refactor of streaming pipeline for python 3.9 support.
• Usage of savers fixed in the threading and processes code after discovering the bug doing the refactor.
• API breaking change toSequentialQueuedPipeline and ParallelQueuedFeatureUnion such that use_threading changed touse_processes to be more clear.
• Some changes to the logging and to objects' print behavior due to the changes in parallelism and python version update.
• Moves the new_trial function to the base HyperparamsRepository class, thus removing duplicated code.
• Improve the implementation of str and repr in BaseStep and in the TruncableSteps.

• Added support for parallel trial execution
• Added separate logger by trial
• Change in setup behaviour
• SKLearnWrapper supports Ensemble methods
• Added Time Series Processing example
• Updated / Cleaned a few example
• And many more quality of life upgrades

Update service assertion interface, extended build to python 3.8, a few minor problems fixed.

Update H1, H2, H3 in intro page

• Fix update_hyperparams for SKLearnWrapper (critical)

• Tree Parzen Estimators
• Scipy Distributions Support

• Partial fit in SKLearnWrapper
• Setup with context
• All the changes from 0.5.1 (was not uploaded to pypi)

• Fix warnings.warn for python 3.6 & python 3.7
• Add with_context method to BaseStep to force the use of a given context in the pipeline
• Use dependency injection by setting the execution context service locator : ExecutionContext().set_service_locator({ BaseType: instance })

• Added streaming pipelines for parallel processing.
• Implemented neat .apply() logic in the steps to recursively apply any function if it exists.
• Refactored the BaseStep so as to make it inherit from multiple smaller classes to separate the logic and allow for creating various steps such as the BaseTransformer that doesn't need any fitting: the BaseStep is thus a BaseTransformer with aditionnal fitting behavior that is added by inheriting another mixin.
• Some bug were fixed.