ludwig | Llama Ecosystem Directory

ludwig - Ludwig v0.10.3 Latest Release

Published by alexsherstinsky 7 months ago

What's Changed

Replace Slack links with Discord links. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3988
Allow image bytes type during preprocessing by @vijayi1 in https://github.com/ludwig-ai/ludwig/pull/3971
Fix for 'upload_to_hf_hub()' path mismatch with 'save()' by @sanjaydasgupta in https://github.com/ludwig-ai/ludwig/pull/3977
Minor change to fix the incorrect response truncation by @amankhandelia in https://github.com/ludwig-ai/ludwig/pull/3986
Pin minimum transformers to 4.39 to reduce Llama/Gemma memory pressure by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3976
Actually add support for RSLoRA and DoRA by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3984

New Contributors

@amankhandelia made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3986

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.10.2...v0.10.3

ludwig - Release v0.10.2

Published by alexsherstinsky 7 months ago

What's New

Add support for RSLoRA and DoRA by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3948
To enable, set the corresponding flag to true in config (can be used in conjunction):

    adapter:
        type:  lora
        use_rslora:  false
        use_dora: false

Add support for eval batch size tuning for LLMs on local backend by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3957
To enable, set "eval_batch_size" to "auto" in the trainer section:

    trainer:
        eval_batch_size:  auto

Enable loading model weights from training checkpoint by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3969
To enable, pass from_checkpoint=True to LudwigModel.load():

LudwigModel.load(model_dir, from_checkpoint=True)

Full Changelog

Save ludwig-config with model-weights in output directory by @sanjaydasgupta in https://github.com/ludwig-ai/ludwig/pull/3965
Add unit tests for image utils unet functions by @vijayi1 in https://github.com/ludwig-ai/ludwig/pull/3921
fix: Update imdb_genre_prediction dataset yaml to match dataset by @jeffreyftang in https://github.com/ludwig-ai/ludwig/pull/3944
Fix kube apt source by @noyoshi in https://github.com/ludwig-ai/ludwig/pull/3952
Temporarily disable expensive text metrics by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3954
[MAINTENANCE] Comment Out PyTorch Nightly Test by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3955
[BUGFIX] Fixing integration test failures. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3959
[MAINTENANCE] Use latest version of psutil library. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3956

New Contributors

@sanjaydasgupta made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3965

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.10.1...v0.10.2

ludwig - Ludwig v0.10.1

Published by alexsherstinsky 8 months ago

What's Changed

fix: use eos token in target tensor for instruction-tuning by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3945

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.10.0...v0.10.1

ludwig - v0.10.0

Published by alexsherstinsky 8 months ago

What's Changed

Add Phi-2 to model presets by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3912
Add default LoRA target modules for Phi-2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3911
Add support for prompt lookup decoding during generation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3917
Pin pyarrow to < 15.0.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3918
Add unet encoder-decoder and image output feature by @vijayi1 in https://github.com/ludwig-ai/ludwig/pull/3913
fix: Add Nested quantization check by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3916
fix typo in save_dequantized_base_model log statement by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3923
Add example for base model dequantization/upscaling by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3924
fix: Always return a list of quantization bits values from get_quantization by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3926
fix: set use_reentrant to True to fix Mixtral-7b bug by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3928
Disabling AdaptionPrompt till PEFT is fixed. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3935
Add default LoRA target modules for Gemma by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3936
Pinning transformers to 4.38.1 or above in order to ensure support for Gemma by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3940
Ludwig release version change by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3941

New Contributors

@vijayi1 made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3913

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9.3...v0.10.0

ludwig -

Published by alexsherstinsky 9 months ago

What's Changed

[MAINTENANCE] Use Trusted Publishers credentials instead of User/Password for uploading releases to PyPi by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3892
Add support for official microsoft/phi-2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3880
Ensure correct padding token for Phi and Pythia models by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3899
Enable AdaLoRA tests for LLM adapter by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3896
Cast LLMEncoder output to torch.float32, freeze final layer at init. by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3900
Enable IA3 adapters in LLMEncoder by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3902
[Maintenance] Remove torch nightly pin by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3903
Pin deepspeed to < 0.13 and pandas to < 2.2.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3906
Add batch size tuning for LLMs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3871

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9.2...v0.9.3

ludwig - v0.9.2: Fixes for OOM and other errors in Ludwig 0.9.1

Published by alexsherstinsky 9 months ago

What's Changed

fix: Handle missing and unexpected keys during LLMEncoder state dict load by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3841
fix: Add name and description classmethods to IA3Config by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3844
Improve IA3 long description by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3845
fix: Handle missing and unexpected keys during LLMEncoder state dict load, part 2 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3843
Update description for max_new_tokens to explain the dynamic setting behavior in our docs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3847
Add default LoRA target modules for Mixtral and Mixtral instruct by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3852
QOL: Fail config validation if a user tries to use ECD with a text output feature and an LLM encoder. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3792
Pin minimum transformers to 4.36 for Mixtral and Phi support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3854
Revert hack that leads to OOM during fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3858
Add support for exporting models to Carton by @VivekPanyam in https://github.com/ludwig-ai/ludwig/pull/3797
[Maintenance] Bump minimum tokenizers to 0.15 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3856
fix: correct typo in FeatureCollection by @dennisrall in https://github.com/ludwig-ai/ludwig/pull/3863
Convert test main script in algorithm_utils to unit test by @dennisrall in https://github.com/ludwig-ai/ludwig/pull/3864
Allow hyperopt config to be loaded from a file by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3865
fix: unify ludwig training set metadata and hf pad token by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3860
Add a utility to detect LLM usage in a config by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3869
Early stop training if model weights have nan or inf tensors by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3740
Scrub credentials from model_hyperparameters.json and description.json by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3866
[Maintenance] Bump minimum torch version to 2.0.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3873
[Maintenance] Fix docker images by pinning ray==2.3.1, daft==0.1.20, unpinning proto, and using torch 2.1.1. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3872
[BUGFIX] Guard against UnicodeEncodeError when saving validation results in Google Colab environment by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3875
Docker image fixes part 2: pin to torch==2.1.0, add dependency for urllib<2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3877
Add custom prepare_for_trianing logic to ECD model for LLM encoder adapter initialization by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3874
qol: Fix some lints. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3868
[Maintenance] Docker Image Fix part 3: fix torchaudio 2.1.0 dependencies by installing libsox-dev and update API by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3879
Add streaming support for zero shot inference by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3878
[Maintenance] Remove torchdata pin for nightly install by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3855
Add per-step token utilization to tensorboard and progress tracker. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3867
Set use_reentrant to False for gradient checkpointing by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3882
[BUGFIX] Pinning torch nightly to January 13, 2024 to avoid AttributeError by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3885

New Contributors

@VivekPanyam made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3797

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9.1...v0.9.2

ludwig - v0.9.1

Published by geoffreyangus 10 months ago

What's Changed

fix: Handle missing and unexpected keys during LLMEncoder state dict load by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3841
fix: Add name and description classmethods to IA3Config by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3844
Improve IA3 long description by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3845
bump ludwig version by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3846

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9...v0.9.1

ludwig - v0.9: Mixtral, Phi, Zephyr, and text classification for LLMs

Published by justinxzhao 10 months ago

What's Changed

int: Rename original combiner_registry to combiner_config_registry, update decorator name by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3516
Add mechanic to override default values for generation during model.predict() by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3520
[feat] Support for numeric date feature inputs by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3517
Add new sythesized response column for text output features during postprocessing by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3521
Disable flaky twitter bots dataset loading test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3439
Add test that verifies that the generation config passed in at model.predict() is used correctly. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3523
Move loss metric to same device as inputs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3522
Add comment about batch size tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3526
Ensure user sets backend to local w/ quantization by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3524
README: Update LLM fine-tuning config by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3530
Revert "Ensure user sets backend to local w/ quantization (#3524)" by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3531
Improve observability during LLM inference by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3536
[bug] Pin pydantic to < 2.0 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3537
[bug] Support preprocessing datetime.date date features by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3534
Remove obsolete prompt tuning example. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3540
Add Ludwig 0.8 notebook to the README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3542
Add effective_batch_size to auto-adjust gradient accumulation by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3533
Refactor evaluation metrics to support decoded generated text metrics like BLEU and ROUGE. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3539
Fix sequence generator test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3546
Revert "Add Cosine Annealing LR scheduler as a decay method (#3507)" by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3545
Set default max_sequence_length to None for LLM text input/output features by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3547
Add skip_all_evaluation as a mechanic to skip all evaluation. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3543
Roll-forward with fixes: Fix interaction between scheduler.step() and gradient accumulation steps, refactor schedulers to use LambdaLR, and add cosine annealing LR scheduler as a decay method. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3555
fix: Move model to the correct device for eval by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3554
Report loss in tqdm to avoid log spam by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3559
Wrap each metric update in try/except. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3562
Move DDP model to device if it hasn't been wrapped yet by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3566
ensure that there are enough colors to match the score index in visua… by @thelinuxkid in https://github.com/ludwig-ai/ludwig/pull/3560
Pin Transformers to 4.31.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3569
Add test to show global_max_sequence_length can never exceed an LLMs context length by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3548
WandB: Add metric logging support on eval end and epoch end by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3586
schema: Add prompt validation check by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3564
Unpin Transformers for CodeLlama support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3592
Add support for Paged Optimizers (Adam, Adamw), 8-bit optimizers, and new optimizers: LARS, LAMB and LION by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3588
FIX: Failure in TabTransformer Combiner Unit test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3596
fix: Move target tensor to model output device in check_module_parameters_updated by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3567
Allow user to specify huggingface link or local path to pretrained lora weights by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3572
Add codellama to tokenizer list for set_pad_token by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3598
Set default eval batch size to 2 for LLM fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3599
[CI] Explicitly set eval batch size in determinism tests, introduce a new integration test group, and exclude slow tests. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3590
[CI] Run sudo apt-get update in GHAs. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3608
Store steps_per_epoch in Trainer by @hungcs in https://github.com/ludwig-ai/ludwig/pull/3601
Updated characters, underscore and comma preprocessors to be TorchScriptable. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3602
[CI] Deflake: Explicitly set eval batch size for mlflow test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3612
Fix registration for char error rate. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3604
fix: Load 8-bit quantized models for eval after fine-tuning by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3606
Add Code Alpaca and Consumer Complaints Datasets by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3611
Add support for gradient checkpointing for LLM fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3613
Bump min support transformers to 4.33.0 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3616
[CI] Fix failing tests on master by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3617
Eliminate short-circuiting for loading from local by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3600
Refactor integration tests into matrix by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3618
fix: Check underlying model device type when moving 8-bit quantized models to GPU at eval by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3622
Fixed range validation for text generation penalty parameters by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3623
Update comment for predict to update Ludwig docs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3535
Avoid deprecation warnings on pandas Series.fillna by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3631
QoL: Default to using fast tokenizer for Llama models by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3625
fixed typo in EfficientNet's model variant from v2_ to v2_s by @saad-palapa in https://github.com/ludwig-ai/ludwig/pull/3628
Add pytorch profiler and additional tensorboard logs for GPU memory usage. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3607
Pin minimum transformers version to 4.33.2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3637
Add function to free GPU memory by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3643
❗ Enable LLM fine-tuning tests when no quantization is specified by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3626
Add check to ensure selected backend works with quantization for LLMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3646
[CI] Use a torch-nightly-compatible version of torchaudio by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3644
Set do_sample default to True by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3641
FIX: Failure in audio feature related test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3651
Remove unnecessary peft config updating by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3642
FIX: docker build error for ludwig-gpu by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3658
Exclude getdaft on Windows by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3629
Add daft back for windows since the wheels are now officially published by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3663
fix: The final batch of an epoch is skipped when batch size is 1 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3653
Place metric functions for BLEU and Rogue on correct devices when using multiple GPUs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3671
Remove duplicate metrics by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3670
Increment epochs based on last_batch() instead of at the end of the train loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3668
[FEATURE] Support Merging LoRA Weights Into Base Model (Issue-3603) by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3649
[FEATURE] Include Mistral-7B model in list of supported base models by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3674
[MAINTENANCE] Partially reconcile type hints, fix some warnings, and fix comments in parts of the codebase. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3673
Improve error message for when an LLM base model can't be loaded. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3675
Fix eos_token and pad_token issue by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3667
FIX: error with nightly CI tests for test_resize_image by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3678
[BUGFIX] Remove spurious test directory at the end of the test_llm.py::test_local_path_loading test run by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3680
Add per-device logging to tensorboard by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3677
Fix dynamic generation config load during model.predict by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3666
[CI] Ensure that mlflow callback cleans up background-saving threads on trainer teardown. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3683
fix: temporarily remove config validation check for backend by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3688
fix: Failing test for backend with quantization by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3689
[BUGFIX] Ensure that full base models and not only adapter weights get saved when merge_and_unload is set by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3679
Add Ludwig Star History to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3696
Use sphinx for all docstrings in api.py by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3693
Fix binary variables being visualized as 0 and 1 by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3691
[MAINTENANCE] Fix the linting warnings in two backend component classes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3698
[BUGFIX] Pin deepspeed<0.11, skip Horovod tests by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3700
Unpin deepspeed following fix in v0.11.1 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3706
Move on_epoch_end and epoch increment to after run_evaluation loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3690
Remove model_load_path from experiment by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3707
[FEATURE] Allow typehints without the quotes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3699
Add consumer complaints generation dataset by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3685
Set the metadata only during first training run by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3684
Add ability to upload Ludwig models to Predibase. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3687
Log additional per-GPU information in model metadata files and GPU utilization on tensorboard. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3712
QoL: Only log generation config being used once at inference time by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3715
[MAINTENANCE] Adding typehint annotations in backend and data components and fixing mypy errors. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3709
QoL: Limit top-level trainer logging messages such as saving model or resuming model training to main coordinator process by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3718
Add sample_size as a global preprocessing parameter by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3650
QOL: Update recommended vscode settings. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3717
Add new fine-tuning notebooks to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3722
Dynamically set max_new_tokens based on output feature length, GMSL and model window size by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3713
Fix issue while logging cuda device utilization to tensorboard by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3727
Pin PEFT to PEFT master for Mistral-7B support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3725
Limit ray versions to keep 2.4 out of the picture. by @trevenrawr in https://github.com/ludwig-ai/ludwig/pull/3731
Set default text preprocessing lowercase behavior to False by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3721
Add functionality to allow direct importing of datasets from Hugging Face by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3664
[MAINTENACE] Improve typehints and use constants instead of common hardcoded string values. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3732
DRAFT: Call .finish() for WandB callback on trainer end. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3710
Fix calculation of dataset size by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3723
[BUGFIX] Make sure that the "hyperopt" directory in the local repository path is deleted after test run is completed by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3736
[BUGFIX] Making integration tests in the group "integration_tests_e" pass in Azure CI by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3737
[CI] Fix test broken by transformers v.4.34.1 by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3739
Raise error if output feature name is in the prompt template by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3743
Retry LLM model downloads from HF Hub with exponential backoff when there is a Read timeout by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3742
Simplify how we set pad token and pad token ID for huggingface tokenizers. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3735
Increment the epoch before running evaluation by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3729
Uncap the torchmetrics version, so we can get fixes in 1.0+. by @trevenrawr in https://github.com/ludwig-ai/ludwig/pull/3741
Fix Consumer Complaints Gen Dataset Config by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3746
[MAINTENANCE] Restrict "fsspec" to not include the latest version "2023.10.0" by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3752
[MAINTENACE] Fixing method overload violations; providing typehints and method argument documentation by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3753
Fix: Prevent memory from ballooning during post-training evaluation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3756
Update base.py by @jetaudio in https://github.com/ludwig-ai/ludwig/pull/3750
Pin getdaft<0.2 in requirements by @jaychia in https://github.com/ludwig-ai/ludwig/pull/3757
QOL: Fail during preprocessing if max sequence lengths are shorter than the prompt template. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3719
Define initialize_llm() and generate() methods. Remove extra logging in llm.py by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3711
Update ludwig version to 0.9.dev. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3638
fix: Check device type in _initialize_llm before moving to cuda. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3762
[BUGFIX] Pin Transformers version to prevent tests from failing in PEFT by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3766
[MAINTENANCE] Update contributor instructions with additional steps that may be needed for installation to succeed. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3764
fix: adapter checkpoint loading on resume by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3769
fix LudwigProgressBar rendering on model resume by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3770
enh: Add on_checkpoint callback that triggers after every checkpoint even if the model was not evaluated by @jeffreyftang in https://github.com/ludwig-ai/ludwig/pull/3763
[BUGFIX] Un-pin "transformers" version and update file names for saved merged model directory, because issue with "transformers" impacting "peft" has been fixed. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3775
OnnxExporter by @skanjila in https://github.com/ludwig-ai/ludwig/pull/3761
[CI] Upgrade torch on CIs to 2.0.0, 2.1.1, and nightly by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3785
Cleanup: Use existing HFTokenizer to consolidate manual pad token setting. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3774
Update copyright by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3778
Add k eval predictions to mlflow for LLM fine-tuning by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3768
Add eval_steps as a flag by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3776
Fix MLFlow logging for sample outputs during evaluation. by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3786
Add NEFTune implementation for Noised Embedding Instruction Fine-Tuning support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3744
[MAINTENANCE] Provide links to contributor guide in README.md by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3794
[Troubleshooting] [BUGFIX] Troubleshooting the "slow" tests group in Azure DevOps CI. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3748
feat: add support for tif files by @dennisrall in https://github.com/ludwig-ai/ludwig/pull/3779
QOL: Fix slow tests GHA by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3795
Added target_modules to LoRA config by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3801
[FEATURE] Generalize LoRA Projection Layers Tests by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3802
Add support for dequantizing 4-bit bitsandbytes base models into fp16 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3799
[Maintenance] Fix deprecation warning for CharErrorRate by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3804
Remove dicts from comparison metrics during visualization by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3796
Fix torch nightly tests: Temporarily add torchdata as a dependency for nightly installation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3808
Log LLM test set evaluation outputs to CSV files under the model log directory, stamped by checkpoint number. by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3793
Introduce CODEOWNERS to ensure at least 1 approval by Ludwig maintainers / the Predibase team by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3809
Add utility function for uploading local folders to huggingface hub by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3805
Fix LLM integration tests that fail because of PEFT 0.7 release by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3810
Add AutoAugmentation to image classification training #3581 by @ethanreidel in https://github.com/ludwig-ai/ludwig/pull/3798
refactor get_total_expected_checkpoint into a function by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3816
Add support for IA3 PEFT Strategy by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3818
Update base LLM preset list to include CodeLlama, Mistral, and Zephyr by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3819
Requirements and import change (ray) by @SarthakNikhal in https://github.com/ludwig-ai/ludwig/pull/3800
Pin Transformers to < 4.36 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3820
Fix epoch computation by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3822
[MAINTENANCE] Support Adapter Model ".safetensors" File Format During HF Upload by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3823
Move reusable LLM model methods to utility functions by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3821
Remove quantization config before saving dequantized weights to disk by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3825
unpin transformers by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3826
[maintenance] small fix in log message for save_dequantized_base_model by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3829
Add support for Phi-1 and Phi 1.5 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3831
hack: workaround torch bug to unblock mixtral fine-tuning by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3830
Add LLM Text Encoder by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3828
pin transformers to patch commit by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3832
enh: Updating requirements to pin huggingface-hub>=0.19.0 by @brightsparc in https://github.com/ludwig-ai/ludwig/pull/3834
Add support for Phi 2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3838
Unpin transformers in favor of 4.36.2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3839
Update LLM trainer defaults by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3840
Update ludwig version to v0.9. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3842

New Contributors

@thelinuxkid made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3560
@saad-palapa made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3628
@alexsherstinsky made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3649
@trevenrawr made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3731
@jetaudio made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3750
@ethanreidel made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3798
@SarthakNikhal made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3800

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8...v0.9

ludwig - v0.8.6

Published by justinxzhao about 1 year ago

What's Changed

Add consumer complaints generation dataset by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3685
Set the metadata only during first training run by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3684
Add ability to upload Ludwig models to Predibase. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3687
Log additional per-GPU information in model metadata files and GPU utilization on tensorboard. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3712
QoL: Only log generation config being used once at inference time by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3715
[MAINTENANCE] Adding typehint annotations in backend and data components and fixing mypy errors. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3709
QoL: Limit top-level trainer logging messages such as saving model or resuming model training to main coordinator process by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3718
Add sample_size as a global preprocessing parameter by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3650
QOL: Update recommended vscode settings. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3717
Add new fine-tuning notebooks to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3722
Dynamically set max_new_tokens based on output feature length, GMSL and model window size by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3713
Fix issue while logging cuda device utilization to tensorboard by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3727

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.5...v0.8.6

ludwig - v0.8.5

Published by justinxzhao about 1 year ago

What's Changed

Add function to free GPU memory by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3643
❗ Enable LLM fine-tuning tests when no quantization is specified by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3626
Add check to ensure selected backend works with quantization for LLMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3646
[CI] Use a torch-nightly-compatible version of torchaudio by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3644
Set do_sample default to True by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3641
FIX: Failure in audio feature related test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3651
Remove unnecessary peft config updating by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3642
FIX: docker build error for ludwig-gpu by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3658
Exclude getdaft on Windows by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3629
Add daft back for windows since the wheels are now officially published by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3663
fix: The final batch of an epoch is skipped when batch size is 1 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3653
Place metric functions for BLEU and Rogue on correct devices when using multiple GPUs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3671
Remove duplicate metrics by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3670
Increment epochs based on last_batch() instead of at the end of the train loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3668
[FEATURE] Support Merging LoRA Weights Into Base Model (Issue-3603) by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3649
[FEATURE] Include Mistral-7B model in list of supported base models by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3674
[MAINTENANCE] Partially reconcile type hints, fix some warnings, and fix comments in parts of the codebase. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3673
Improve error message for when an LLM base model can't be loaded. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3675
Fix eos_token and pad_token issue by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3667
FIX: error with nightly CI tests for test_resize_image by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3678
[BUGFIX] Remove spurious test directory at the end of the test_llm.py::test_local_path_loading test run by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3680
Add per-device logging to tensorboard by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3677
Fix dynamic generation config load during model.predict by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3666
[CI] Ensure that mlflow callback cleans up background-saving threads on trainer teardown. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3683
fix: temporarily remove config validation check for backend by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3688
fix: Failing test for backend with quantization by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3689
[BUGFIX] Ensure that full base models and not only adapter weights get saved when merge_and_unload is set by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3679
Add Ludwig Star History to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3696
Use sphinx for all docstrings in api.py by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3693
Fix binary variables being visualized as 0 and 1 by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3691
[MAINTENANCE] Fix the linting warnings in two backend component classes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3698
[BUGFIX] Pin deepspeed<0.11, skip Horovod tests by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3700
Unpin deepspeed following fix in v0.11.1 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3706
Move on_epoch_end and epoch increment to after run_evaluation loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3690
Remove model_load_path from experiment by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3707
[FEATURE] Allow typehints without the quotes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3699

New Contributors

@alexsherstinsky made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3649

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.4...v0.8.5

ludwig - v0.8.4

Published by justinxzhao about 1 year ago

What's Changed

Add codellama to tokenizer list for set_pad_token by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3598
Set default eval batch size to 2 for LLM fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3599
[CI] Explicitly set eval batch size in determinism tests, introduce a new integration test group, and exclude slow tests. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3590
[CI] Run sudo apt-get update in GHAs. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3608
Store steps_per_epoch in Trainer by @hungcs in https://github.com/ludwig-ai/ludwig/pull/3601
Updated characters, underscore and comma preprocessors to be TorchScriptable. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3602
[CI] Deflake: Explicitly set eval batch size for mlflow test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3612
Fix registration for char error rate. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3604
fix: Load 8-bit quantized models for eval after fine-tuning by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3606
Add Code Alpaca and Consumer Complaints Datasets by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3611
Add support for gradient checkpointing for LLM fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3613
Bump min support transformers to 4.33.0 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3616
[CI] Fix failing tests on master by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3617
Eliminate short-circuiting for loading from local by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3600
Refactor integration tests into matrix by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3618
fix: Check underlying model device type when moving 8-bit quantized models to GPU at eval by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3622
Fixed range validation for text generation penalty parameters by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3623
Update comment for predict to update Ludwig docs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3535
Avoid deprecation warnings on pandas Series.fillna by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3631
QoL: Default to using fast tokenizer for Llama models by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3625
fixed typo in EfficientNet's model variant from v2_ to v2_s by @saad-palapa in https://github.com/ludwig-ai/ludwig/pull/3628
Add pytorch profiler and additional tensorboard logs for GPU memory usage. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3607
Pin minimum transformers version to 4.33.2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3637

New Contributors

@saad-palapa made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3628

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.3...v0.8.4

ludwig - v0.8.3

Published by justinxzhao about 1 year ago

What's Changed

Add test to show global_max_sequence_length can never exceed an LLMs context length by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3548
WandB: Add metric logging support on eval end and epoch end by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3586
schema: Add prompt validation check by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3564
Unpin Transformers for CodeLlama support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3592
Add support for Paged Optimizers (Adam, Adamw), 8-bit optimizers, and new optimizers: LARS, LAMB and LION by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3588
fix: Failure in TabTransformer Combiner Unit test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3596
fix: Move target tensor to model output device in check_module_parameters_updated by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3567
Allow user to specify huggingface link or local path to pretrained lora weights by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3572

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.2...v0.8.3

ludwig - v0.8.2

Published by justinxzhao about 1 year ago

What's Changed

int: Rename original combiner_registry to combiner_config_registry, update decorator name by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3516
Add mechanic to override default values for generation during model.predict() by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3520
[feat] Support for numeric date feature inputs by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3517
Add new sythesized response column for text output features during postprocessing by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3521
Disable flaky twitter bots dataset loading test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3439
Add test that verifies that the generation config passed in at model.predict() is used correctly. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3523
Move loss metric to same device as inputs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3522
Add comment about batch size tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3526
Ensure user sets backend to local w/ quantization by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3524
README: Update LLM fine-tuning config by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3530
Revert "Ensure user sets backend to local w/ quantization (#3524)" by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3531
Revert "Ensure user sets backend to local w/ quantization" for release-0.8 branch and upgrade version to 0.8.1.post1 by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3532
Improve observability during LLM inference by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3536
[bug] Pin pydantic to < 2.0 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3537
[bug] Support preprocessing datetime.date date features by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3534
Remove obsolete prompt tuning example. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3540
Add Ludwig 0.8 notebook to the README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3542
Add effective_batch_size to auto-adjust gradient accumulation by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3533
Refactor evaluation metrics to support decoded generated text metrics like BLEU and ROUGE. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3539
Fix sequence generator test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3546
Revert "Add Cosine Annealing LR scheduler as a decay method (#3507)" by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3545
Set default max_sequence_length to None for LLM text input/output features by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3547
Add skip_all_evaluation as a mechanic to skip all evaluation. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3543
Roll-forward with fixes: Fix interaction between scheduler.step() and gradient accumulation steps, refactor schedulers to use LambdaLR, and add cosine annealing LR scheduler as a decay method. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3555
fix: Move model to the correct device for eval by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3554
Report loss in tqdm to avoid log spam by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3559
Wrap each metric update in try/except. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3562
Move DDP model to device if it hasn't been wrapped yet by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3566
ensure that there are enough colors to match the score index in visua… by @thelinuxkid in https://github.com/ludwig-ai/ludwig/pull/3560
Pin Transformers to 4.31.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3569

New Contributors

@thelinuxkid made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3560

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.1...v0.8.2

ludwig - v0.8.1.post1

Published by justinxzhao about 1 year ago

What's Changed

Revert "Ensure user sets backend to local w/ quantization" for release-0.8 branch and upgrade version to 0.8.1.post1 by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3532

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.1...v0.8.1.post1

ludwig - v0.8.1

Published by arnavgarg1 about 1 year ago

What's Changed

Update ludwig version to 0.8.1 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3527
Release 0.8.1 latest by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3528

Commits

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8...v0.8.1

ludwig - v0.8: Low Code Framework to Efficiently Build Custom LLMs on Your Data

Published by tgaddair about 1 year ago

Full Release Blog Post Here: https://predibase.com/blog/ludwig-v0-8-open-source-toolkit-to-build-and-fine-tune-custom-llms-on-your-data

What's Changed

Make fill_value a medium impact parameter in preprocessing by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3155
Allow auto cherry-pick into release-0.7 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3157
Fixed confidence_penalty for newer versions of pytorch by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3156
Fixed set explanations by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3160
Bump to hummingbird 0.4.8 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3164
Update SequenceGeneratorDecoder to output predictions and probabilities by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3152
Disable sampling in preprocessing or when it results in too few rows by @ShreyaR in https://github.com/ludwig-ai/ludwig/pull/3117
Unpin pyarrow by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3167
Make Horovod an optional dependency when using Ray by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3166
Skip sample_ratio validation when using Dask to prevent materialization of DF by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3174
Fix TorchVision channel preprocessing by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3173
Bump Ludwig to v0.7.1 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3179
Add fallback mirrors to dataset API by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3168
Log cached dataset write paths during cache miss by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3181
Re-enable benchmark tests on Sarcos dataset by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3169
Disable passthrough decoder for all feature types by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3151
Log when cached dataset can't be found by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3192
Remove hard dependency on ludwig[tree]. Check model.type() instead of instanceof(model). by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3184
Add sequence decoder integration tests by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3175
int: [REBASE] Remove unnecessary JSON schema code by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3196
fix: [REBASE] Hoist uniqueItemProperties to top of feature JSON schema by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3183
Guarantee determinism when sampling (either overall via sample_ratio, or while balancing data) by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3191
Use tmpdir for more files generated during tests by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3197
Removed .vscode and added to .gitignore by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3201
Revert vscode by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3202
Fixes dict_hash discrepancy by @w4nderlust in https://github.com/ludwig-ai/ludwig/pull/3195
Reset Ray address by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3200
Unpin scikit-learn. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3185
Fixed learning_rate_scheduler params in automl by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3203
Fix Docker image dependencies and add tests for minimal install by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3186
Bump to v0.7.2 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3208
fix: [REBASE] Misc. JSON schema fixes by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3187
fix: [REBASE] Streamline GBM defaults schema by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3188
Update sequence/text feature max_sequence_length default to None by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3205
Add category onehot encoder for both ECD and GBM by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3057
Enable transformer encoder and disable embed encoder from SequenceCombiner by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3154
Add auxiliary validation for all features to be present in comparator combiner entities by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3216
Disable number decoder in the decoder config by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3217
Add non_zero to common_fields.NumFCLayersField by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3215
Add the ability to specify a local or S3 HF mirror for more guaranteed loading of pre-trained HF models. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3211
Schemafy merge_fixed_preprocessing_params by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3223
Tagger decoder config override and auxiliary validation checks by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3222
Refactor loss implementation to use the schema config for all parameters by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3227
follow up: unregister passthrough decoder for all features by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3225
top_k should be a positive integer by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3230
Refactored combiner registry and broke circular dep with schema by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3228
Implements sequence_length param by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3221
Add columns and data types to ludwig datasets by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3231
Fixes Explain step for tied weights by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3214
Fix @slow hf tests by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3233
Remove partial RayTune checkpoints for trials that have not completed because of forceful termination by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3232
Replace NaN in timeseries rows with padding_value by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3238
Adds support for TEXT features when using GBM with tf-idf encoder by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3235
Persist Dask Dataframe after binary image/audio reads by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3241
Add Timeseries forecasting for column-major data, and introduce Timeseries output feature by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3212
fix: transform onehot encoder outputs to float32 tensor by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3242
Pin torchaudio by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3244
Pin torchvision and torchtext by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3248
Filter entities from comparator combiner when not listed in input_features by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3251
Reset os.environ var in hf_utils. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3253
fix typo in pretrained_model_name_or_path by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3257
Fixes torch DDP distributed metric computation for AUROC by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3234
Unpin torchvision, torchtext, and torchaudio. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3255
Added compute tiers to parameter metadata by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3254
Allow encoders for GBMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3258
feat: Add env var LUDWIG_SCHEMA_VALIDATION_POLICY to change marshmallow validation strictness by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3226
Removes invalid keys from GBM defaults in the schema by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3252
Add support for model.compile in PyTorch 2.0 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3246
Fix ludwig docker by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3264
Bump to v0.7.3 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3267
Update gpt text encoder afn parameter default to what's listed in HF docs. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3261
Add test for falling back to HF model that's not in the ludwig pretrained dir by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3256
Fixed non-scalar (text, vector, set) output feature explanations by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3269
Use LudwigFeatureDict to permit module keys that are rejected by torch ModuleDict. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3270
Fixed handling of datetime types in input parquet files by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3274
Fix date explanations by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3276
Handle np.bool to JSON NumpyEncoder by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3280
Update all combiners to use .get to access LudwigFeatureDict contents by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3279
Check that concat combiner doesn't receive a mixture of non-reduced sequence and non-sequence features. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3271
Better handling for missing dataset columns by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3285
feat: Add kwargs option to all file readers and feed nrows where possible by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3266
Update version to 0.8.dev by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3286
Added DeBERTa (v2 / v3) text encoder by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3289
Check fill_with_const has fill_value for binary features by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3278
Fixed reduce_option=concat for auto_transformer and deberta by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3291
Adds category_distribution output feature for providing labels as a probability distribution by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3288
fix: [REBASE] Add config transformation that removes extra type param if it exists in the defaults config by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3296
Pass in kwargs to read_parquet by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3293
Unpin transformers by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3290
fix: Fix read_parquet for remote filesystems by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3294
fix: Remove duplicate enums in deberta parameters by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3299
fix: Add separate output config registries for ECD and GBM by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3306
Use frac arg in df.sample instead of n by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3307
Added prompt_template preprocessing param for text features by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3298
Allow different splits for hyperopt metric reporting by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3282
fix: Raise error if list provided to StringOptions has duplicates and change validation errors to assertion errors for this field as well. by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3302
Add PROBABILITIES and PREDICTIONS to the prediction set for the sequence tagger decoder. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3300
Fixed URI loading by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3310
Added trainer.gradient_accumulation_steps for increasing effective batch size by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3305
Fixed cache_encoder_embeddings with tied weights by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3308
test: Parameterize dataset read chunking test by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3311
Allow providing encoders / decoder params as strings by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3309
Fixes for CI failures by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3315
Pin pandas < 2.0 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3319
Add combinatorial tests to the CI by @abidwael in https://github.com/ludwig-ai/ludwig/pull/2991
int: Prevent encoder enums from being reshuffled by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3313
Ensure that iq and all number feature preprocessing normalizations work on dask backends. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3297
Back out dataset profile protos in ludwig. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3318
Docker image upgrades to Python 3.9, Ray 2.3.1, Torch 2.0.0, CUDA 11.8. Deprecating Horovod by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3320
Consolidate hyperopt schema in ludwig.schema.hyperopt by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3190
Downgrade docker images to use Python 3.8 by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3325
Prevent GBM memory leaks by forcing actor cleanup after each epoch by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3323
Pass encoding dimensions to SequenceCombiner by @RXminuS in https://github.com/ludwig-ai/ludwig/pull/3321
Filter auto_transformer kwargs based on forward signature by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3329
Strip dataset path to figure its format correctly by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3331
Use Python 3.8 in pytest by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3330
Added union on ModelConfig and dict in create_model by @aseemk98 in https://github.com/ludwig-ai/ludwig/pull/3333
Speed up average number of tokens computation by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3337
Fix sequence decoder attribute error by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3336
Decode bytes to strings for DatasetInfo avg_words by @hungcs in https://github.com/ludwig-ai/ludwig/pull/3338
Add compute tier for DeBERTa by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3340
Pandas 2.0 update by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3322
fix image values inference for FieldInfo by @hungcs in https://github.com/ludwig-ai/ludwig/pull/3341
Add input and output shapes for attention reducer (combiner) by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3339
Added distinct decoder registries by model type by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3342
Skip ames tests by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3345
[LLM] Support zero-shot learning and text generation through LLMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3335
Pin dask<2023.4.0 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3347
Fix dependency install in CI by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3354
fix slow tests by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3327
Bump Transformers to 4.28.1 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3357
CI install dependency refactor by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3355
Use dist.barrier instead of horovod legacy code by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3358
Fixed distributed strategy registration to be explicit by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3361
Tune batch size using distributed training to catch edge case CUDA OOMs by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/2934
Allow checkpoint download on non-coordinator process by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3363
Description Fix by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3364
Add accuracy with micro averaging for category features. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3367
temporary pip install expecttest for torch nightly tests by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3366
Clean up distributed batch size tuning logging by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3370
Ensure worker synchronization during resume by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3369
Remove expecttest temporary install by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3371
Added DeepSpeed distributed strategy and backend by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3362
Default to LocalStrategy for GBM metrics by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3373
Implemented CORN loss for ordinal classification by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3375
Fix MNIST datasets img path column by @shawnccx in https://github.com/ludwig-ai/ludwig/pull/3376
[LLM] Few-shot learning via Retrieval-augmented ICL by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3351
Correct TorchScript Typo in Readme by @samster25 in https://github.com/ludwig-ai/ludwig/pull/3380
Temporarily switch to eval mode when using batch_size: 1 by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3378
Fixed DDP checkpointing to save to local disk on each node by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3381
Add special handling for empty ray datasets. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3384
Ensure captum's halved batch size retry fn wrapper accepts the minimum batch size of 1. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3382
[LLM] Fine-Tune LLMs via Prompt tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3359
Added LoRA tuner for HuggingFace text encoders by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3385
fix: Typo in zscore metadata by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3388
Skip broken image torchvision test with efficientnet. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3389
fix: Fix dataset reading for parquet directories by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3356
Cut CI runtime down to 1 hour by splitting integration tests into 3 separate runs. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3391
Remove the flatten/unflatten ray backend tensor communication workaround by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3301
revert: Revert "fix: Fix dataset reading for parquet directories (#3356)" by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3396
Sanitize GBM feature names to remove JSON special characters by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3326
Add back in support for Dictionary Class Weights by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3392
fix: sequence feature explain error by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3397
[LLM] Add Prefix Tuning, PTuning, LoRA, AdaLoRA and Adaption Prompt for LLM fine-tuning by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3386
[llm] Move prompt into top-level of config by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3399
[LLM] Set additional properties to False for Adapter field by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3401
[llm] Added support for bfloat16 with deepspeed by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3403
[llm] Parameter metadata by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3404
Introduce a fourth test grouping to further speed up integration tests. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3405
Ray nightly compatibility by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3275
[LLM] Fix Loss Computation for LLM Fine-tuning using shifted tensors/new loss function by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3408
Fix tokenizer pad and eos token assumption for HF tokenizers by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3411
Revert "Ray nightly compatibility (#3275)" by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3413
[llm] Create separate Predictor for LLMs and enable flash attention on CUDA by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3409
Fix Missing Param Metadata by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3415
[llm] Fixed adapter initialization and OOM on checkpoint loading by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3416
Remove Mutable Defaults by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3418
Fix: Device placement for metrics during evaluation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3420
Order prompt schema and add title by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3419
[llm] Fixed loading when performing full fine-tuning by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3421
[llm] Use lower default learning rate and rename llm decoders to extractors by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3422
Remove the other None Param Metadata imputation by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3424
Add model name param metadata by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3425
Support combining multiple columns into a single text feature by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3426
fix: Install html5lib and test html reads by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3428
Added vector index interface and fixed import errors by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3429
[LLM] Skip left padding removal when there is no left padding by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3432
Update README.md, fixed some quotes by @neuhausler in https://github.com/ludwig-ai/ludwig/pull/3436
Improve image/audio read throughput by 50% for image/audio features using Daft by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3249
[LLM] Various fixes for LLM Fine-Tuning issues that caused loss disparity between train and val sets by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3437
int: Add back required for input and output features to the Ludwig JSON schema by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3442
int: Pin getdaft to 0.1.6 by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3443
upgrade ludwig-gpu docker image to pytorch 2.0.0 by @jppgks in https://github.com/ludwig-ai/ludwig/pull/3445
[llm] Replace model_name with required base_model, add preset LLM registry, update internal adapter modules by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3423
[llm] fix device placement issues when using CPUs and GPUs during LLM fine tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3447
Unpin Daft by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3451
Add param metadata to column parameter by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3452
fix: Add titles for augmentation schema options by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3454
pin torchmetrics to 0.11.4 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3456
Adding --no-cache-dir to dockerfile pip install by @noyoshi in https://github.com/ludwig-ai/ludwig/pull/3455
Replace all non-word characters in feature names to ensure no downstream issues with external libraries. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3438
[Example] Fix llm_finetune example json part by @chongxiaoc in https://github.com/ludwig-ai/ludwig/pull/3461
refactor: add type hints for encoders by @Dennis-Rall in https://github.com/ludwig-ai/ludwig/pull/3449
Unpin deepspeed by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3466
Unpin torch nightly CI. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3459
typo in parameter of README.md by @Skizzy-create in https://github.com/ludwig-ai/ludwig/pull/3471
Zero copy initialization of models onto training workers for LLMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3469
Remove tables requirement as it causes issues installing ludwig in linux env. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3473
Add QLoRA for 4-bit fine-tuning by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3476
int: Fix quantization schema by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3479
Fixed divide by zero when tuning batch size by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3481
Exclude frozen weights from checkpoints and fix evaluation using quantized LLMs by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3483
Removed extra clone during prediction and added quantization error handling by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3484
Updated Ludwig CI for torch nightly by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3467
Fixed QLoRA with mutli-gpu and reduce CUDA memory pressure during eval by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3486
Enable evaluation with LLMs <7B by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3478
Add ludwig upload to push artifacts to HuggingFace Hub by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3480
Add RoPE scaling to increase context length up to 8K for training or inference. by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3477
Added quantization parameter metadata by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3487
Added CLI llama2 example by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3491
Updated presets with Llama-2 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3490
[fix] Multiple test fixes by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3489
fix: Read partitioned parquet files from relative paths by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3470
Require using lora adapter when performing quantized fine-tuning by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3492
Require text output feature for LLM finetuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3493
Pin bitsandbytes<0.41.0 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3494
int: Pin sqlalchemy to 1.x.x versions by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3496
Add parameter metadata for global_max_sequence_length by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3497
Remove falcon from presets as it requires running untrusted code by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3495
Updates for ludwig-docs by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3499
Add use_pretrained attribute for AutoTransformers by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3498
Readme updates for 0.8 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3500
Improve description for generation config parameters by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3501
Readme TYPO by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3502
Make Ludwig logo smaller in the README by @abidwael in https://github.com/ludwig-ai/ludwig/pull/3505
Check that LLMs have exactly one text input feature by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3508
Fix temperature description by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3509
Add Cosine Annealing LR scheduler as a decay method by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3507
Fix typo in function name for LR schedulers by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3511
Bump to v0.8 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3512

New Contributors

@RXminuS made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3321
@aseemk98 made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3333
@shawnccx made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3376
@samster25 made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3380
@neuhausler made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3436
@chongxiaoc made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3461
@Skizzy-create made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3471
@Infernaught made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3467

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.7...v0.8

ludwig - v0.7.5

Published by justinxzhao about 1 year ago

What's Changed

Fixed URI loading by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3314
Filter auto_transformer kwargs based on forward signature by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3332
Remove tables for Ludwig 0.7. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3503
Update ludwig version to v0.7.5 by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3506

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.7.4...v0.7.5

ludwig - v0.7.4

Published by justinxzhao over 1 year ago

What's Changed

Tagger decoder config override and auxiliary validation checks (#3222)

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.7.3...v0.7.4

ludwig - v0.7.3

Published by tgaddair over 1 year ago

What's Changed

Support for PyTorch 2.0 via trainer.compile: true (#3246)
Fix ludwig docker (#3264)
Add env var LUDWIG_SCHEMA_VALIDATION_POLICY to change marshmallow validation strictness (#3226)
Add sequence_length capability (#3259)
Persist Dask Dataframe after binary image/audio reads (#3241)
Replace NaN in timeseries rows with padding_value (#3238)
Remove partial RayTune checkpoints for trials that have not completed because of forceful termination (#3232)

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.7.2...v0.7.3

ludwig - v0.7.2

Published by tgaddair over 1 year ago

What's Changed

Fix Docker image dependencies and add tests for minimal install (#3186)
Fixed learning_rate_scheduler params in automl (#3203)
Fixes dict_hash discrepancy (#3195)
Guarantee determinism when sampling (either overall via sample_ratio, or while balancing data) (#3191)
Add sequence decoder integration tests (#3175)
Update SequenceGeneratorDecoder to output predictions and probabilities (#3152)
Remove hard dependency on ludwig[tree]. Check model.type() instead of instanceof(model) (#3184)

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.7.1...v0.7.2