Low-code framework for building custom LLMs, neural networks, and other AI models
APACHE-2.0 License
Published by w4nderlust over 3 years ago
Published by w4nderlust almost 4 years ago
-f
to -ofn
(#1032)Published by w4nderlust almost 4 years ago
Published by w4nderlust about 4 years ago
hyperopt
command.auto
, csv
, df
, dict
, excel
, feather
, fwf
, hdf5
(cache file produced during previous training), html
(file containing a single HTML <table>
), json
, jsonl
, parquet
, pickle
(pickled Pandas DataFrame), sas
, spss
, stata
, tsv
.batch_predict
functionality in the REST API.collect_summary
command to print out a model summary with layers names.predict
command, and splitt it into predict
and evaluate
. The first only produces predictions, the second evaluates those predictions against ground truth.hyperopt_report
and hyperopt_hiplot
.This release includes a fundamental rewrite of the internals, so many bugs have been fixed while rewiting.
This list includes only the ones that have a specific Issue associated with them, but many others where addressed.
Because of the change in the underlying tensor computation library (TensorFlow 1 to TensorFlow 2) and the internal reworking it required, models trained with v0.2 don't work on v0.3.
We suggest to retrain such models, in most cases the same model definition can be used, although one impactuful breaking change is that model_definition
are now called config
, because they don't contain only information about the model, but also training, preprocessing, and a newly added hyperopt section.
There have been some changes in the parameters inside the config too.
In particular, one main change is dropout
that now it is a float value that can be specified for each encode / combiner / decoder / layer, while before it was a boolean parameter.
As a consequence, the dropout_rate
parameter in the training section has been removed.
Another change in training parameters are the available optimizers.
TensorFlow 2 doesn't ship with some of the ones that were exposed in Ludwig (adagradda
, proximalgd
, proximaladagrad
) and the momentum optimizer has been removed as now it is a parameter of the sgd
optimizer.
Newly added optimizers are nadam
and adamax
.
Note that the accuracy
metric for the combined
feature has been removed because it was misleading in some scenarios when multiple features of different types where trained.
In most cases, encoders, combiners and decoders now have an increased number of exposed parameters to play with for increased flexibility.
One notable change is that the previous BERT encoder has been replaced by an HuggingFace based one with different parameters, and it is now available only for text features.
Please refer to the User Guide for details for each encoder.
Tokenizers also changed substantially with new parameters supported, refer to User Guide for more details.
Other major changes are related to the CLI interface.
The predict
command has been replaced in functionality with a simplified predict
and a new evaluate
. The first only produces predictions, the second evaluates those predictions against ground truth.
Some parameters of all CLI commands changed.
All different type of data_*
parameters have been replaced by generic dataset
, training_set
, validation_set
and test_set
parameters, while the data format is automatically determined, but can also be set manually by using the data_format
argument. There is no
gpu_fractionany more, but now users can specify
gpu_limit` for managing the VRAM usage.
For all additional minor changes to the CLI please refer to the User Guide.
The programmatic API changed too, as a consequence.
Now all the parameters match closely the ones of the CLI interface, including the new dataset
and gpu
parameters.
Also in this case the predict
function has been split into predict
and evaluate
.
Finally, the returned values of most functions changed to include some intermediate processing values, like for instance the preprocessed and split data when calling train
, the output experiment directory and so on.
Notably, now there is an experiment
function in the API too, together with a new hyperopt
one.
For more datails, refer to the API reference.
@jimthompson5802 @tgaddair @kaushikb11 @ANarayan @calio @dme65 @ydudin3 @carlogrisetti @ifokeev @flozi00 @soovam123 @KushalP1 @JiByungKyu @stremlau @adiov @martinremy @dsblank @jakobt @vkuzmin-uber @mbzhu1 @moritzebeling @lnxpy
Published by w4nderlust over 4 years ago
Added integration with Weights and Biases.
Added K-Fold cross validation.
Added 4 examples with their respective code and Jupyter Notebooks: Hyper-parameter optimization, K-Fold Cross Validation, MNIST, Titanic.
Greatly improved the measures tracked on the TensorBoard.
Added auto-detect function for field separator when reading CSVs.
Added CI tooling.
Class weights can be specified as a dictionary #615.
Removed deprecation warning from h5py.
Removed most deprecation warning from TensorFlow.
Bypass multiprocessing.Pool.map for faster execution.
Updated TensorFlow dependency to 1.15.2.
Various documentation improvements.
Fix cudnn error on RTX GPUs.
Fix inverted confusion_matrix axis.
Fix #201: Removed whitespace as a separator option.
Fix #540: Fixed default text parameters for sampled loss.
Fix #541: Docker image improvements (removed libgmp and spacy model download).
Fix #554: Fix audio input test case in docker container.
Fix #570: Temporary dolution for in_memory
flag usage in API.
Fix #574: Setting intra and inter op parallelism to 0 so that TF determine them automatically.
Fix #329 and #575: Fixed use of SavedModel and added an integration test.
Fix #609: When predicting, if a split is in the CSV, data is split correctly.
Fix #616: Change preprocessing in siamese network example.
Fix #620: Failure in unit tests for 1 vs all calibration plots.
Fix #632: Setting minimum version requirements for six
.
Fix #636: CLI output table column ordering preserved when resuming.
Fix #641: Added multi-task learning section specifying the weight for each output feature in the User Guide.
Fix #642: Fixing horovod use when loading a model as initialization.
@jimthompson5802 @calz1 @pingsutw @vanpelt @carlogrisetti @anttisaukko @dsblank @borisdayma @flozi00 @jshah02
Published by w4nderlust about 5 years ago
Add Filter Bank features for audio.
Added two more parameters skip_save_test_predictions
and skip_save_test_statistics
to train and experiment CLI commands and API.
Updated to spaCy 2.2 with support for Norvegian and Lithuanian tokenizers.
Reorganized dependencies, now the defaults are barebone and there are several axtra ones.
Added fc_layers
to H3 embed encoder.
Added get_preprocessing_params
in preprocessing.
Refactored image features preprocessing to use multiprocessing.
Refactored preprocessing with strategy pattern.
Fix #452: Removed dependency on gmpy
.
Fix #465: Adds capability to set the vocabulary from a Glove file.
Fix #480: Adds a health check to ludwig serve
.
Fix #481: Added some examples of visualization commands.
Fix #491: Improved skip parameters, now no directories are created if not needed.
Fix #492: Adds skip saving unprocessed output api.py
.
Fix #493: Added parameters for the vocabulary file and the UNK
and PAD
symbols in sequence feature call to create_vocabulary
in the calculation of metadata.
Fix #500: Fixed learning_curves()
when the training statistics file does not contain validation.
Fix #509: Fixes in_memory
issues in image features.
Fix #525: Adding check is_on_master()
before creating save_path
dir./ectory
Fix #510: Fixed version of pydantic.
Fix #532: Improved speed of add_sequence_feature_column()
.
Fix #520: Renamed field parameter in visualization to output_feature_name for clarity and improved documentation. Please make sure to rename you function calls if you were using this parameter by name (the order keeps the same).
@sriki18 @carlogrisetti @areeves87 @naresh-bhandari @revolunet @patrickvonplaten @Athanaziz @dsblank @tgaddair @Mechachleopteryx @AlexeyGy @yu-iskw
Published by w4nderlust over 5 years ago
New BERT encoder and with its BPE tokenizer
Added Audio features that can be used also for speech data (with appropriate preprocessing feature extraction)
Added H3 feature, together with 3 encoders to deal with spatial information
Added Date feature and two encoders to deal with temporal information
Improved Comet.ml integration
Refactored visualization.py
to make individual functions usable from API
Added capability of saving visualization graph in the visualization command and visualizations_utils.py
Added a serve
command that allows for spawning a prediction server using FastAPI
Added a test
command (that requires output columns in the data) to avoid confusion with predict
(which does not require output columns)
Added pixel normalization and pixel standardization scaling options for image features
Added greyscaling of images if specified channels = 1 and img channels is 3 or 4
Added normalization strategies for numerical features (#367)
Added experiment name parameter in the API (#357)
Refactored text tokenizers
Several improvements in logging
Added a method for saving models with SavedModels
in model.py
and exposes it in the API with a save_for_serving()
function (#329)(#425)
Upgraded to the latest version of TensorFlow 1.14 (#429)
Added learning rate warmup for non distributed settings
Fix #321: Removed the 6n+2 check for ResNet size
Fix #328: adds missing UPDATE_OPS to the optimization operation
Fix #336: GloVe embeddings loading now reads utf-8 encoded files
Fix #336: Addresses the malformed lines issue in embeddings loading
Fix #346: added a parameter indicating if the session should be closed after training in full_train
Fix #351: values in categorical columns are now stripped before being compared to the vocabulary
Fix #364: associate the right function to non english text format functions
Fix #372: set evaluate performance parameter to false in predict.py
Fix #394: Improved error explaination when image dimensions don't match and improved documentation accordingly
Fix #411: Images in HDF5 are now correctly saved as uint8
instead of int8
Fix #431: missing libgmp3-dev dependency in docker (#428)
Fix fixed image resizing
Fix model load path (#424)
Fix batch norm in convolutional layers (now uses tf internal layer and not the one in contrib)
Several additional minor fixes
@carlogrisetti @jaipradeesh @glongh @dsblank @danicattaneob @gogasca @lordeddard @IgorWilbert @patrickvonplaten @ojus1 @jimthompson5802 @johnwahba @revolunet @gogasca
Published by w4nderlust over 5 years ago
only_predict
with evaluate_performance
(and flipped the logic) in all predict commands and functionsdata_dict
to the train method in LudwigModel
word_format
in text features is now properly usedyaml.safe_load
instead of yaml.load
and replaced pickling of the progress tracker with a JSON equivalenttied_weights
key in some featuresLudwigModel
would be imported from ludwig
now it should be imported from ludwig.api
. This change was needed for speeding up imports@dsblank @Ignisor @bertyhell @jaipradeesh
Published by w4nderlust over 5 years ago
skip_save_progress
, skip_save_model
and skip_save_log
parametersPassthroughEncoder
eval_batch_size
parameterThanks to all our amazing contributors (some of your PRs were not merged, but we used some of their code in our commits, so thank you anyway!):
@dsblank @MariusDanner @BenMacKenzie @Barathwaja @gabefair @kevinqz @yantsey @jontonsoup4 @Praneet460 @DakshMiglani @syeef @Tejaf @rolisz @JakeConnors376W @AndyZZH @us @0xflotus @laserbeam3 @krychu @dettmering @bbrodsky @c-m-hunt @C0deFxxker @hemchander23 @Shivam-Beeyani @yashrajbharti @rbramwell @emushtaq @EBazarov @graytowne @jovilius @ivanhe @philippgille @floscha
Published by w4nderlust over 5 years ago
This is the first public release of Ludwig