This repository is the source code for fine tuning any LLM in just one line 🔥
APACHE-2.0 License
Fine-tune any Large Language Model (LLM) available on Hugging Face in a single line.
one-line-llm-tuner
is a Python package designed to simplify the process of fine-tuning large language models (LLMs) like GPT-2, Llama-2, GPT-3 and more. With just one line of code, you can fine-tune a pre-trained model to your specific dataset. Consider it as a wrapper for transformers
library, just like how keras
is for tensorflow
.
transformers
library, including GPT, BERT, and more.You can install one-line-llm-tuner
using pip:
pip install one-line-llm-tuner
The PyPI package can be used in the following way after installation.
from one_line_llm_tuner.tuner import llm_tuner
fine_tune_obj = llm_tuner.FineTuneModel()
fine_tune_obj.fine_tune_model(input_file_path="train.txt")
fine_tune_obj.predict_text("Elon musk founded Spacex in ")
The above example is for default values. If you want to modify the default values such as type of model used, tokenizer and more, use the following code.
from one_line_llm_tuner.tuner import llm_tuner
fine_tune_obj = llm_tuner.FineTuneModel(model_name="gpt2",
test_size=0.3,
training_dataset_filename="train_dataset.txt",
testing_dataset_filename="test_dataset.txt",
tokenizer_truncate=True,
tokenizer_padding=True,
output_dir="./results",
num_train_epochs=2,
logging_steps=500,
save_steps=500,
per_device_train_batch_size=128,
per_device_eval_batch_size=128,
max_output_length=100,
num_return_sequences=1,
skip_special_tokens=True,)
fine_tune_obj.fine_tune_model(input_file_path="train.txt")
fine_tune_obj.predict_text("Elon musk founded Spacex in ")
We welcome contributions! Please see the contributing guide for more details.
This project is licensed under the terms of the Apache license. See the LICENSE file for details.