Convert Pytorch to Paddle Toolkit
APACHE-2.0 License
This Repo contains the toolkit that help you transform the pytorch model to paddle model. eg: Weight file Comparer, Weight Converter, Weight Summary, and so on ...
cppt gen_diff
: generate the diff between paddle weight and torch weight file, for more details you check out ...cppt auto_match
: auto match the names of paddle and torch occording to the name semantic, eg: torch.model.embeddings.embed_token.weight -> paddle.opt.model.word_embeddings.weight ...cppt convert
: convert the torch weight file to paddle weight file according to the diff file ...cppt summary
: summary the tensor meta info according to the weight files and diff file ...In order to convert pytorch weight file to paddle weight file and make sure that the logits of paddle model is absolute align with pytorch, there are some steps you should follow.
cppt gen_diff
to find the diffs between layer names.cppt auto_match
command to auto match the names with similarity algo. you can edit the final diff file, and make it correct.cppt convert
command to convet the pytorch weight file to paddle weight file. In this abstract, the script will automaticly transpose the linear-weight tensor.cppt summary
the generate the meta info between paddle weight file.So, it's cool right ? these codes help me converting
work more soft.
But, there are also some great works can be done:
If you have more ideas about it, you can post issue to discuss with us. We look forward to discussing it with you.
pip install paddle-cppt
or install from the source code:
git clone https://github.com/wj-Mcat/paddle-cppt
cd paddle-cpp
python setup.py install
If you complete the code, you can init the paddle weight file with the following code:
from paddlenlp.transformers.{your-model}.modeling import {YourModel}
# this code will be different according to different model, but anyway the final result is to save the state dict of model which contains the layer names of your model code.
model = {YourModel}(
model={YourModel}(
**{YourModel}.pretrained_init_configuration['{name-of-configuration}']
)
)
model.save_pretrained('/path/to/local/dir')
this command will generate the name mapping
cppt gen_diff \
--torch_file=/path/to/pytorch_model.bin \
--paddle_file=/path/to/model_state.pdparams \
--output_file=/path/to/diff.xlsx --auto_match
with this command, you will get the diff.xlsx
file which contains the layer name of paddle/torch weight file, eg:
torch-name | torch-shape | torch-dtype | torch-type | paddle-name | paddle-shape | paddle-dtype | paddle-type |
---|---|---|---|---|---|---|---|
embeddings.position_ids | [1, 512] | torch.int64 | embedding | embeddings.word_embeddings.weight | [30522, 768] | paddle.float32 | embedding |
embeddings.word_embeddings.weight | [30522, 768] | torch.float32 | embedding | embeddings.position_embeddings.weight | [512, 768] | paddle.float32 | embedding |
embeddings.position_embeddings.weight | [512, 768] | torch.float32 | embedding | embeddings.token_type_embeddings.weight | [2, 768] | paddle.float32 | embedding |
embeddings.token_type_embeddings.weight | [2, 768] | torch.float32 | embedding | embeddings.layer_norm.weight | [768] | paddle.float32 | embedding |
embeddings.LayerNorm.weight | [768] | torch.float32 | embedding | embeddings.layer_norm.bias | [768] | paddle.float32 | embedding |
embeddings.LayerNorm.bias | [768] | torch.float32 | embedding | encoder.layers.0.self_attn.q_proj.weight | [768, 768] | paddle.float32 | linear-weight |
encoder.layer.0.attention.self.query.weight | [768, 768] | torch.float32 | linear-weight | encoder.layers.0.self_attn.q_proj.bias | [768] | paddle.float32 | linear-bias |
encoder.layer.0.attention.self.query.bias | [768] | torch.float32 | linear-bias | encoder.layers.0.self_attn.k_proj.weight | [768, 768] | paddle.float32 | linear-weight |
encoder.layer.0.attention.self.key.weight | [768, 768] | torch.float32 | linear-weight | encoder.layers.0.self_attn.k_proj.bias | [768] | paddle.float32 | linear-bias |
encoder.layer.0.attention.self.key.bias | [768] | torch.float32 | linear-bias | encoder.layers.0.self_attn.v_proj.weight | [768, 768] | paddle.float32 | linear-weight |
encoder.layer.0.attention.self.value.weight | [768, 768] | torch.float32 | linear-weight | encoder.layers.0.self_attn.v_proj.bias | [768] | paddle.float32 | linear-bias |
encoder.layer.0.attention.self.value.bias | [768] | torch.float32 | linear-bias | encoder.layers.0.self_attn.out_proj.weight | [768, 768] | paddle.float32 | linear-weight |
encoder.layer.0.attention.output.dense.weight | [768, 768] | torch.float32 | linear-weight | encoder.layers.0.self_attn.out_proj.bias | [768] | paddle.float32 | linear-bias |
encoder.layer.0.attention.output.dense.bias | [768] | torch.float32 | linear-bias | encoder.layers.0.linear1.weight | [768, 3072] | paddle.float32 | linear-weight |
encoder.layer.0.attention.output.LayerNorm.weight | [768] | torch.float32 | norm | encoder.layers.0.linear1.bias | [3072] | paddle.float32 | linear-bias |
this command will generate the final name mapping into the excel file.
cppt auto_match \
--diff_file=/path/to/diff.xlsx \
--output_file=/path/to/diff-result.xlsx
convert torch model to paddle weight according the final diff file.
cppt convert \
--torch_file=/path/to/pytorch_model.bin \
--output_file=/path/to/model_state.pdparams \
--diff_file=/path/to/diff-result.xlsx
print the summary metadata info between torch and paddle model
cppt summary \
--torch_file=/path/to/pytorch_model.bin \
--output_file=/path/to/model_state.pdparams \
--diff_file=/path/to/diff.xlsx
Released under Apache License 2.0 by @wj-Mcat.