PaddleNLP

**** | English

PaddleNLP(LLM)PaddleNLP ********

News

• **2024.08.08 PaddleNLP 3.0 **8221900https://www.wjx.top/vm/Y2f7FFY.aspx?udsid=143844

• **2024.06.27 PaddleNLP v3.0 Beta**4D RsLoRA+ Unified Checkpoint FastFFNFusedQKV

• 2024.04.24 PaddleNLP v2.8 RsLoRA+ PEFT RLHF PPO PPO PPO FastFFNFusedQKV

GPU XPU NPU GCU DCU

4D Trainer Unified Checkpoint

FlashMask

• LLaMA Baichuan Bloom ChatGLM Gemma Mistral OPT Qwen LLM

• 4D LLaMA Baichuan Bloom ChatGLM Gemma Mistral OPT Qwen LLM4D

• SFTPEFT LLaMA Baichuan Bloom ChatGLM Mistral OPT Qwen LLM

• LLaMA Qwen Mistral ChatGLM Bloom Baichuan Weight Only INT8 INT4 WACCache KV INT8FP8LLM

• python >= 3.8
• paddlepaddle >= 3.0.0b0

PaddlePaddle

pip

pip install --upgrade paddlenlp==3.0.0b2

develop

pip install --pre --upgrade paddlenlp -f https://www.paddlepaddle.org.cn/whl/paddlenlp.html

PaddlePaddle PaddleNLP Installation

PaddleNLP Auto API Tokenizer Qwen/Qwen2-0.5B

>>> from paddlenlp.transformers import AutoTokenizer, AutoModelForCausalLM
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-0.5B")
>>> model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B", dtype="float16")
>>> input_features = tokenizer("", return_tensors="pd")
>>> outputs = model.generate(**input_features, max_length=128)
>>> print(tokenizer.batch_decode(outputs[0], skip_special_tokens=True))
['AI']

git clone https://github.com/PaddlePaddle/PaddleNLP.git && cd PaddleNLP # clonePaddleNLP
mkdir -p llm/data && cd llm/data
wget https://bj.bcebos.com/paddlenlp/models/transformers/llama/data/llama_openwebtext_100k.bin
wget https://bj.bcebos.com/paddlenlp/models/transformers/llama/data/llama_openwebtext_100k.idx
cd .. # change folder to PaddleNLP/llm
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_pretrain.py ./config/llama/pretrain_argument.json

SFT

git clone https://github.com/PaddlePaddle/PaddleNLP.git && cd PaddleNLP # clonePaddleNLP
mkdir -p llm/data && cd llm/data
wget https://bj.bcebos.com/paddlenlp/datasets/examples/AdvertiseGen.tar.gz && tar -zxvf AdvertiseGen.tar.gz
cd .. # change folder to PaddleNLP/llm
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py ./config/llama/sft_argument.json

PaddleNLP

• PaddleNLP NLP
• AI Studio PaddleNLP

• .

Citation

PaddleNLP

@misc{=paddlenlp,
    title={PaddleNLP: An Easy-to-use and High Performance NLP Library},
    author={PaddleNLP Contributors},
    howpublished = {\url{https://github.com/PaddlePaddle/PaddleNLP}},
    year={2021}
}

Acknowledge

Hugging Face Transformers Hugging Face

License

PaddleNLP Apache-2.0