New models

Highlight features

1. Dataset refactoring, to be compatible with HF datasets structure, new to limit datasets<2.19.0 (Breaking Changes)

What's Changed

• Fix api get_dataset_file_url by @wangxingjun778 in https://github.com/modelscope/modelscope/pull/811
• Feature/raft by @pipikk in https://github.com/modelscope/modelscope/pull/702
• fix installation error in README_zh.md by @cyx2000 in https://github.com/modelscope/modelscope/pull/821
• BUGFIX: TypeError: exceptions must derive from BaseException by @joeyism in https://github.com/modelscope/modelscope/pull/816
• add eres2netv2 by @yfchenmodelscope in https://github.com/modelscope/modelscope/pull/830
• add input output for siamese by @WeiLai5432 in https://github.com/modelscope/modelscope/pull/810
• Fix datasets hf utils by @wangxingjun778 in https://github.com/modelscope/modelscope/pull/834
• merge from release 1.13.3 by @zzhangpurdue in https://github.com/modelscope/modelscope/pull/837

New Contributors

• @pipikk made their first contribution in https://github.com/modelscope/modelscope/pull/702
• @cyx2000 made their first contribution in https://github.com/modelscope/modelscope/pull/821
• @joeyism made their first contribution in https://github.com/modelscope/modelscope/pull/816
• @WeiLai5432 made their first contribution in https://github.com/modelscope/modelscope/pull/810

Full Changelog: https://github.com/modelscope/modelscope/compare/v1.13.2...v1.14.0

Highlight features

1. Dataset refactoring, to be compatible with HF datasets structure. (Breaking Changes)
2. Unified datasets storage and management with GIT. (Breaking Changes)

What's Changed

• upload marigold monocular depth estimation core files by @Ranqing in https://github.com/modelscope/modelscope/pull/703
• chore: Formal LICENSE content by @tisonkun in https://github.com/modelscope/modelscope/pull/799
• add ViViT-demo by @ccyhxg in https://github.com/modelscope/modelscope/pull/796
• move doc to classroom by @tastelikefeet in https://github.com/modelscope/modelscope/pull/802
• Fix error "modelscope attributeerror: 'dict' object has no attribute … by @wertycn in https://github.com/modelscope/modelscope/pull/800
• fix download file with spical name as 'Image+Title.png' by @liuyhwangyh in https://github.com/modelscope/modelscope/pull/805
• Dataset refactor by @wangxingjun778 in https://github.com/modelscope/modelscope/pull/807

New Contributors

• @Ranqing made their first contribution in https://github.com/modelscope/modelscope/pull/703
• @tisonkun made their first contribution in https://github.com/modelscope/modelscope/pull/799
• @wertycn made their first contribution in https://github.com/modelscope/modelscope/pull/800

Full Changelog: https://github.com/modelscope/modelscope/compare/v1.13.1...v1.13.2

New models

Highlight features

1. Support importing AWQConfig from modelscope
2. Support stream_generate for LLMPipeline

What's Changed

• UViT ImageNet by @ccyhxg in https://github.com/modelscope/modelscope/pull/763
• Add DiT_ImageNet_Demo by @ccyhxg in https://github.com/modelscope/modelscope/pull/767
• Add a QA example based on llamaindex (with qwen1.5) by @yhxx511 in https://github.com/modelscope/modelscope/pull/759
• Fix words by @co63oc in https://github.com/modelscope/modelscope/pull/747
• Fix word pubicly by @co63oc in https://github.com/modelscope/modelscope/pull/748
• Fix word pipleline by @co63oc in https://github.com/modelscope/modelscope/pull/749
• Fix word orignal by @co63oc in https://github.com/modelscope/modelscope/pull/750
• add awqconfig by @Jintao-Huang in https://github.com/modelscope/modelscope/pull/761
• add SiT_ImageNet_Demo by @ccyhxg in https://github.com/modelscope/modelscope/pull/770
• Fix words StableDiffuisonExporter -> StableDiffusionExporter by @co63oc in https://github.com/modelscope/modelscope/pull/746
• add self supervised depth completion. by @heyyxd in https://github.com/modelscope/modelscope/pull/711
• To solve the "ImportError: always import a name 'LlamaTokenizer' from… by @Siu-Ming in https://github.com/modelscope/modelscope/pull/745
• Fix code of conduct by @tastelikefeet in https://github.com/modelscope/modelscope/pull/774
• add res2net resnet models by @yfchenmodelscope in https://github.com/modelscope/modelscope/pull/772
• upgrade image build by @liuyhwangyh in https://github.com/modelscope/modelscope/pull/773
• change output video format from mp4v to mp4 h264 by @qslia in https://github.com/modelscope/modelscope/pull/757
• fix text_to_video_synthesis_model device by @slin000111 in https://github.com/modelscope/modelscope/pull/751
• add qwen1.5_doc_search_QA_based_on_langchain.ipynb by @ccyhxg in https://github.com/modelscope/modelscope/pull/769
• Add files via upload by @ccyhxg in https://github.com/modelscope/modelscope/pull/788
• pre-commit passed by @tastelikefeet in https://github.com/modelscope/modelscope/pull/789
• fix build issue by @liuyhwangyh in https://github.com/modelscope/modelscope/pull/786
• fix SeqGPTPipeline input force cuda issue by @RainJayTsai in https://github.com/modelscope/modelscope/pull/738
• fix image_portrait_enhancement_pipeline rgb channel issue by @sun11 in https://github.com/modelscope/modelscope/pull/740
• remove download interval check by @liuyhwangyh in https://github.com/modelscope/modelscope/pull/771
• Fix initilize initialize, etc by @co63oc in https://github.com/modelscope/modelscope/pull/781
• fix pre-commit by @tastelikefeet in https://github.com/modelscope/modelscope/pull/794
• Features/cv_geomvsnet_multi_view_depth_estimation_general by @shengzhesz in https://github.com/modelscope/modelscope/pull/790
• Support stream_generate for LLMPipeline by @Firmament-cyou in https://github.com/modelscope/modelscope/pull/768

New Contributors

• @heyyxd made their first contribution in https://github.com/modelscope/modelscope/pull/711
• @Siu-Ming made their first contribution in https://github.com/modelscope/modelscope/pull/745
• @yfchenmodelscope made their first contribution in https://github.com/modelscope/modelscope/pull/772
• @qslia made their first contribution in https://github.com/modelscope/modelscope/pull/757
• @RainJayTsai made their first contribution in https://github.com/modelscope/modelscope/pull/738
• @sun11 made their first contribution in https://github.com/modelscope/modelscope/pull/740
• @shengzhesz made their first contribution in https://github.com/modelscope/modelscope/pull/790

Full Changelog: https://github.com/modelscope/modelscope/compare/v1.12.0...v1.13.1

中文版本

新模型推荐

高亮功能

• add rife-video-frame-interpolation and model (#685)
• image normal estimation (#683)
• add image matching fast model based on lightglue (#694)
• Feature/LoFTR_image_local_feature_matching (#687)
• support qwen1.5 models
• upgrade funasr1.0

BugFix

• fix anydoor pre-commit flake8 and isort errors (#707)
• fix some ci case issue.

New Models Recommended

Highlight

• Add AnyDoor support (#688)
• Add syncdreamer as a image-to-3d pipeline (#679)
• Add audio codec and codec-based TTS model

Improvements

• Update cuda to 12.1.0
• Update transformers to 4.36.2
• Update ckpt to general_v0.1 (#696)

BugFix

• Fix embedding and inference device in faq question answering pipeline
• Remove DOCKER_BUILDKIT=0 for cpu build issue
• Fix mmcv-full issue

中文版本

新模型推荐

高亮功能

• 支持本地拉起测试推理服务
• 支持vllm推理
• LLMPipeline 支持vllm
• 官方镜像升级到python3.10，pytorch升级2.1.0, tensorflow 1.14.0 ubuntu22.04
• upgrade to python3.10

功能列表

• Support VLLM in LLMPipeline (#604)
• add bpemodel path in asr_trainer
• add llm riddles (#621)
• feat: deploy checker for swingdeploy

功能提升

• python311 support for whl
• llm pipeline support chatglm3 (#618)
• Support transformers==4.35.0 (#633)

BugFix

• Fix _set_gradient_checkpointing bug (#660)
• fix test reliability issue (#657)
• fix: DocumentGroundedDialogRetrievalModel qry_encoder.encoder.embeddings.position_ids error (#647)
• fix asr paraformer finetune bug
• fix uie trainer: eval failed (#617)
• Fix vllm: change if condition (#607)
• fix shop_segmentation to use old timm lib and bump version to 1.9.4rc2
• fix the numpy bug for card detection correction
• fix issues for 3dhuman models
• fix logger: remove file handler for original user logging (#645)

English Version

Highlight

• local launch inference server
• support vllm
• LLMPipeline support vllm
• Image upgrade to python3.10, pytorch2.1.0,tensorflow2.14.0, ubuntu22.04

Breaking changes

Feature

• Support VLLM in LLMPipeline (#604)
• add bpemodel path in asr_trainer
• add llm riddles (#621)
• feat: deploy checker for swingdeploy

Improvements

• python311 support for whl
• llm pipeline support chatglm3 (#618)
• Support transformers==4.35.0 (#633)

BugFix

• Fix _set_gradient_checkpointing bug (#660)
• fix test reliability issue (#657)
• fix: DocumentGroundedDialogRetrievalModel qry_encoder.encoder.embeddings.position_ids error (#647)
• fix asr paraformer finetune bug
• fix uie trainer: eval failed (#617)
• Fix vllm: change if condition (#607)
• fix shop_segmentation to use old timm lib and bump version to 1.9.4rc2
• fix the numpy bug for card detection correction
• fix issues for 3dhuman models
• fix logger: remove file handler for original user logging (#645)

中文版本

Feature

• 新增句子向量模型，支持gte, bloom
• stable diffusion新增freeU方法
• LLMPipeline 支持Swift adapter模型推理
• 镜像制作时自动升级funasr transformer最新版本
• venv强制依赖移除，以便更好地支持windows系统 #575

bugfix

• 修复 shop_segmentation pipeline兼容timm 0.5.2
• 修复huggingface position_ids兼容性问题
• 修复chatglm sp_tokenizer属性确实问题
• 修复ofa模型transformers新版兼容性问题
• 修复trainer中work_dir设置不生效问题 #573
• 修复hf相关的bug #569 #567

新增模型推荐

English Version

Feature

• Added sentence vector model, supporting gte and bloom.
• Stable diffusion introduces a new freeU method.
• LLMPipeline now supports Swift adapter model inference.
• Automatically upgrade to the latest version of funasr transformer during image creation.
• Forced venv dependency removed to better support Windows system. #575

bugfix

• Fixed shop_segmentation pipeline compatibility with timm 0.5.2.
• Resolved compatibility issues with huggingface position_ids.
• Fixed the missing sp_tokenizer attribute in chatglm.
• Addressed compatibility issues of ofa model with newer transformers version.
• Resolved the issue where the work_dir setting in trainer was not taking effect. #573
• Fixed hf-related bugs. #569 #567.

New Models Recommended

中文版本

高亮功能

• 优化ci
• 兼容transformers 4.34.0
• Support int4 model for llm_pipeline

BugFix

• fix merge error (#582)
• move venv import from file level to class level to avoid import error… (#575)

English Version

Highlight

• optimize ci
• compatible with transformers 4.34.0
• Support int4 model for llm_pipeline

BugFix

• fix merge error (#582)
• move venv import from file level to class level to avoid import error… (#575)

中文版本

新模型推荐

高亮功能

• 增加image_control_3d_portrait模型
• 增加3dhuman render and animation 模型
• 增加LLMPipeline支持大模型推理

功能列表

• 支持swift trainer和pipeline
• 增加image_control_3d_portrait模型
• 增加3dhuman render and animation 模型
• 新增 model for card correction
• 新增 head_reconstruction and text_to_head model
• 增加LLMpipeline支持大模型推理
• 增加 onnx exporter for ocr recognition model
• 增加 onnx exporter for ocr_detection db model

功能提升

BugFix

• 修复onnxruntime 新版本兼容性问题
• 修复huggingface兼容性问题
• asr支持本地模型
• 修复ci问题

English Version

New Model List and Quick Access

Highlight

• Add 3dhuman render and animation models
• Add image_control_3d_portrait
• Add LLMpipeline support LLM inference

Breaking changes

Feature

• support swift trainer and pipeline (#547)
• add image_control_3d_portrait
• add 3dhuman render and animation models
• add model for card correction
• add head_reconstruction and text_to_head model
• add LLMpipeline support LLM inference
• add onnx exporter for ocr recognition model
• add onnx exporter for ocr_detection db model

Improvements

BugFix

• Fix onnxruntime providers parameter compatible issue
• Fix hf bug (#569)
• Fix support local asr models (#556)
• Fix fix ci issue

中文版本

新模型推荐

高亮功能

• 模型下载增加失败重试功能

功能列表

功能提升

• 支持模型下载失败重试

BugFix

• 解决新版本transformers position_ids兼容性问题

English Version

New Model List and Quick Access

Highlight

• Retry download model when failed.

Breaking changes

Feature

• Retry download model when failed.

Improvements

BugFix

• Fix latest transformers position_ids compatible issue.

中文版本

新模型推荐

高亮功能

• video to video model support a10, v100
• funasr支持mossformer模型
• SDXL支持lora微调
• stable diffusion支持fp16训练和推理

不兼容更新

• 最低支持python3.8
• 镜像tensorflow 升级2.13.0
• numpy，pandas版本升级

功能列表

功能提升

• 更新qwen QA 示例
• 添加qwen QA langchain示例

BugFix

• 修复stable diffusion fp16 bug
• 修复图像上色模型加载问题
• 修复 chatglm2b rope_ratio config 缺失问题

English Version

New Model List and Quick Access

Highlight

• video to video model support a10, v100
• Add funasr support mossformer model
• Support sdxl finetune by lora method
• support float16 training and pipeline for stable diffusion

Breaking changes

• Deprecated python3.7 support
• tensorflow upgrade to 2.13.0 in image
• numpy, pandas version upgrade

Feature

Improvements

• upgrade qwen QA sample

BugFix

• Fix bugs of stable diffusion fp16
• fix image colorization model load issue
• fix chatglm2b rope_ratio config is missing

中文版本

新模型

VideoComposer：组合视频合成 (#431)

改进

• 移除numpy版本 <=1.22.0 的限制 (#453)
• 更改llama2的max_length默认值 (#452)
• 在函数generate中支持llama2输入到设备
• 支持为llama加载数据集
• 更新qwen qa示例
• 添加readme和警告 (#462)

Bug修复

• 修复chatglm2b rope_ratio配置参数缺失 (#440)
• 修复pipeline检查错误 (#455)
• 修复针对python37的copytree bug (#464)

English Version

New models

VideoComposer: Compositional Video Synthesis with Motion Controllability (#431)

Improvement

• remove restriction of numpy version <=1.22.0 (#453)
• change llama2 max_length default value (#452)
• support llama2 inputs to device in function generate
• support load dataset for llama
• update qwen qa example
• add readme and warning (#462)

Bugfix

• fix chatglm2b rope_ratio config is missing (#440)
• fix pipeline check error (#455)
• fix copytree python37 bug (#464)

bugfix for qwen

• streamer pass error
• check flash attention installation even if use_fast_att is set True
• fix quantization model run failed

中文版本

新模型推荐

功能列表

• 训练时使用AutoModel情况下增加模型版本检查
• 支持safe tensors weight pipeline。
• 支持transformers类模型的流式输出。
• 优化类包装器。
• 支持huggingface transformers的AutoModel, AutoConfig and AutoTokenizer
• 在llm中添加了完整参数的sft。
• 添加了日语README。
• 在params和load函数中添加了download_mode参数。
• 使用Xformers 提高了attention计算时的内存/显存性能。

功能提升

• 将 stable diffusion 版本升级到更强大的版本2.1。
• 增加custom stable diffusion 微调。
• 添加了 stable diffusion swift tuner。
• 在不使用deepspeed的情况下支持llama和lora微调。
• 为lora  stable diffusion 添加了lora_rank参数。
• 优化了torch1.11和torch2.0.1镜像构建脚本。
• 在sbert文本分类中支持从数据集获取标签，并在chatglm-6b中构建文件数据集。
• 更新了speaker_verification_pipeline.py。
• 更新了默认测试级别。
• 更新了aliyuncs pip的默认值。
• 更新了语言识别任务名称。
• 为text-to-video添加了height和width参数。
• 自定义 diffusion pipeline。
• 添加了支持ASRDataset的download_mode参数。
• 更新了chatglm6b v2的新版本。
• 改进了加载meta-csv缓存路径的方式。
• 添加了一个example/llm模块。
• 优化了注释和格式。
• 设置download_mode的默认值。
• 使用num_inference_steps和guidance_scale参数更新pipeline。
• 在finetune_speech_recognition.py中添加了download_mode的支持，使用params.download_mode。
• 在finetune_speech_recognition.py中使用ASRDataset替换MsDataset。
• 更新了ASRDataset，为重新下载数据集（如数据集损坏或损坏）添加了download_mode。
• 更新了asr_dataset.py，支持使用download_mode重新下载数据。
• 将text_in设置为必需参数。
• 修改了text_generation_pipeline类的参数传递。
• 添加了baichuan/chatglm2+lora+agent示例。
• 添加了 stable diffusion 教程ipynb。

BugFix

• 忽略http错误，以防止模型检查时混淆。
• 解决了加载 checkpoint时出现的不同设备问题。
• 修复了缺少plugin模块文件的问题。
• 解决了镜像标签无cuda的问题。
• 解决了easycv CPU扩展构建问题。
• 解决了设备错误问题。
• 解决了amp和device_map问题。
• 通过配置pysptk >= 0.1.19解决了pip安装错误。
• 解决了ckpt输出目录忽略*.safetensors的问题。
• 修复了对baichuan的eval和sequence_length支持的问题。
• 解决了在文档分割pipeline中使用cuda设备的问题。
• 删除了清华相关的硬编码。
• 解决了与stable diffusion pipeline相关的错误，该pipeline无法识别“lora_scale”参数。
• 解决了与chatglm2模块相关的错误。
• 修复了mPLUG-Owl生成长度错误。
• 更正了speaker模型的详细信息。
• 修复了在加载本地stable diffusion数据集时出现的错误。
• 修复了chatglm管道中的错误。
• 修复了chatglm6b 2的错误。
• 修复了与empty hypothesis相关的chatglm2评估错误。

English Version

New Model List and Quick Access

Feature

• Added check model for training and model_dir for automodel.
• Added support for safetensors weight pipeline.
• Support for stream output on transformers model
• Refined class wrapper
• Added Support for AutoModel, AutoConfig and AutoTokenizer
• Added full parameter sft to llm
• Added Japanese README
• Added download_mode param to params and load function.
• Accelerated memory efficient attention with Xformers.

Improvements

• Upgraded stable diffusion version to more powerful version 2.1.
• Custom method for finetuning stable diffusion
• Added stable diffusion swift tuner
• Support for llama & lora finetune without deepspeed
• Added lora_rank parameter for lora stable diffusion
• Refactored torch1.11 and torch2.0.1 build script
• Support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b
• Updated speaker_verification_pipeline.py
• Changed tests level
• Set default aliyuncs pip
• Added height and width parameters for text-to-video.
• Customized the diffusion pipeline.
• Added support for ASRDataset for download_mode parameters.
• Updated the Fea/chatglm6b v2 new version.
• Improved the load meta-csv cathe paths.
• Added an example/llm module.
• Optimized comments and formatting.
• Set the download_mode default value.
• Updated the pipeline with num_inference_steps and guidance_scale parameters.
• Added support for download_mode in finetune_speech_recognition.py with params.download_mode.
• Replaced MsDataset with ASRDataset in finetune_speech_recognition.py.
• Updated ASRDataset with download_mode for re-downloading the dataset if it is broken or corrupted.
• Updated asr_dataset.py to support download_mode for re-downloading data.
• Made text_in a required parameter.
• Modified the parameter passing of the text_generation_pipeline class.
• Added baichuan/chatglm2 +lora+agent examples.
• Added stable diffusion tutorial ipynb.

BugFix

• Ignored http error to prevent confusion during model check.
• Fixed checkpoint issue related to same device.
• Fixed missing plugin python module files.
• Fixed build tag no cuda issue.
• Fixed easycv CPU extension build issue.
• Fixed device error.
• Fixed issue with amp and device_map
• Fixed pip install error with pysptk>=0.1.19
• Fixed issue with ckpt output directory ignoring *.safetensors
• Fixed eval and sequence_length support for baichuan
• Fixed issue with using cuda device in document segmentation pipeline inference
• Removed hard code tsinghua
• Fixed amp + device_map (#386)
• Fixed bugs related to the Chinese stable diffusion pipeline not recognizing the 'lora_scale' argument.
• Fixed bugs related to the chatglm2 module.
• Fixed mPLUG-Owl generating length bug.
• Fixed details of speaker models.
• Fixed bugs related to loading local sd dataset.
• Fixed bugs in the chatglm pipeline.
• Fixed chatglm6b 2.
• Fixed chatglm2 evaluation error related to empty hypothesis.

Release 1.7.2
Fix some bugs

中文版本

新特性

• 增加baichuan模型lora inference
• 增加baichuan和chatglm2 lora agent示例

BugFix

• 修复历史问题
• 修复sd加载本地数据集的问题
• 修复chatglm推理问题
• 修改text_generation_pipeline传参
• 修复chatglm6b和chatglm6b 2
• 修复评估报错：hypothesis emtpy

English Version

Features

• Add lora_inference for baichuan
• Add baichuan/chatglm2+lora+agent examples

BugFix

• fix history problem
• Fix a bug of loading local stable diffusion dataset
• Fix/chatglm pipeline
• Modify the parameter passing of the text_generation_pipeline class
• Fix/chatglm6b 2
• Fix/chatglm6b
• fix evaluation error: hypothesis emtpy

中文版本

新模型推荐

高亮功能

• 新增轻量化端侧识别模型LightweightEdge
• 新增轻量化端侧DBNet行检测模型
• 新增CAM++说话人转换点定位
• llama模型支持finetune和deepseed
• llama模型支持lora
• 对于transformers模型支持device_map
• 数据集支持jsonl格式
• 对于大型模型文件支持并行下载（dsw或eais环境）
• 提升youku超大型数据集下载体验

功能列表

• 新增轻量化端侧识别模型LightweightEdge
• 新增轻量化端侧DBNet行检测模型
• 新增flextrain ner样例
• 新增在文本分类finetune的training args中增加模型版本
• 新增StreamingMixin
• 支持torch extension
• 支持llama模型微调和deepspeed
• pipeline中支持第三方的key
• 支持说话人分离pipeline
• 新增eres2net_aug v2模型
• 支持transformers model device_map
• 支持模型权重diff
• 数据集支持jsonl格式
• 新增Lora/Adapter/Prompt/Chatglm6b
• 部分tests增加teardown
• 解除datasets包版本限制
• 支持从model id加载
• llama模型支持lora
• 对于大型模型文件支持并行下载

功能提升

• 提升mPLUG-youku大型数据集的下载体验

BugFix

• 修复DeepspeedHook.register_processor
• dockerfile的兼容性修改（py37和py38）
• 修复 extra_args
• 修复ngpu bug和移除easyasr
• 修复mplug-youku超大数据集下载相关问题
• 修复gpt3 finetune nan的问题
• 修复torch extension ci hang住的问题
• 修复easycv lr hook 错误
• 修复torch2.x 兼容性问题
• 修复diffusers版本冲突问题
• 修复eval RecursionError
• 对于DiffusionForTextToImageSynthesis修复device_map问题
• 修复stable diffusion pipeline cpu推理问题
• 修复llama lora问题

English Version

New Model List and Quick Access

Highlight

• Add new OCR recognition model (LightweightEdge) and some functions
• Add ocr detection new model db-nas
• Add CAM++ model
• Support llama model finetune and deepspeed
• Support lora for llama model
• Support device_map for transformers
• Support jsonl format in datasets
• Support parallel download large model file
• Improve mPLUG-YOUKU dataset downloading experience

Breaking changes

Feature

• Add new OCR recognition model (LightweightEdge) and some functions
• Add ocr detection new model db-nas
• Add ner example for flextrain
• Add model revision in training_args and modify dataset loading in finetune text classification
• Add StreamingMixin
• Support pre build torch extension build image, first extension megatron_util
• Add llama finetune + deepspeed
• Support third_party key in pipeline
• Add speaker diarization pipeline and improve some speaker pipelines
• Add eres2net_aug v2
• Support device_map for transformers model
• Add make diff & recover for model weights
• Support jsonl format in meta data
• Add Lora/Adapter/Prompt/Chatglm6b
• Add teardown for tests
• Unfreeze datasets version setting
• Support load from model id
• Support lora for llama
• Support parallel download large model file

Improvements

• llama tuned model -> pipeline
• improve youku dataset downloading experience

BugFix

• Fix bug for DeepspeedHook.register_processor
• Docker file py38 and py37 compatible merge
• Fix extra_args
• ngpu bug and rm easyasr
• Fix issues for downloading mplug-youku dataset
• Fix gpt3 finetune nan
• Fix ci hang when build torch extension
• Fix easycv lr hook error
• Fix torch 2.x compatible issue
• Fix diffuser version conflict cv and multi-modal
• Fix eval RecursionError
• Fix device_map for DiffusionForTextToImageSynthesis
• Fix cpu inference for stable diffusion pipeline
• Fix llama lora bug

中文版本

功能列表

• 支持跳过easycv三方依赖引入
• 支持Flextrain training args和push_to_hub
• 支持domain_specific_object_detection 的onnx格式导出

BugFix

• 修复test_cli CI报错
• 修复merge hook
• 修复NER tokenizer不能接收kwargs的问题
• 修复lineless_table_recognition功能遇到空白图片崩溃的bug
• 修复某些情况下private数据集鉴权失败的问题

English Version

Feature

• Add pattern to skip easycv.thirdparty
• Support flex train feature (training args and push_to_hub adaptions)
• Support onnx export for domain_specific_object_detection

BugFix

• Fix CI: test merge dataset failed
• Fix merge_hook
• Fix NER tokenizer which won't accept kwargs
• Fix lineless_table_recognition crashed when input blank images
• fix private dataset auth issue

中文版本

该版本共新增上架5个模型。

新模型列表及快捷访问

非兼容性修改

• 支持Python3.8版本
• 移除demo check

English Version

Highlight

• Support Python3.8
• Add mPLUG-Owl model
• Add cvpr23 Fastinst model

Breaking changes

• Support Python3.8
• Remove demo check

Feature

• Add ERes2Net for speaker verification
• Add mPLUG-Owl model
• Support FlexTrain and update the structure of trainer
• Add cvpr23 fastinst model
• Support Virgo MaxCompute datasource for Ali-cloud inner applications
• Add clip_interrogator
• Add gpt3 example
• Add convert megatron ckpt script
• Add trainer for UniTE
• Add transface model
• Add verified if whl installed
• Support python3.8
• Add ONNX exporter for ans dfsmn
• Add rdino model

Improvements

• Update multi_modal_embedding example
• Refine easrasr
• Pipeline input, output and parameter normalization.
• Display hub error message
• Remove easycv codes, plugin access

BugFix

• Fix bug in **kwargs duplicated for audio module
• Fix distributed hook to lazyimport and an import bug
• Fix transformer examples
• Add pop for base class parameters
• Fix func update_local_model; change funasr version
• Remove pai-easycv requirement
• Fix hypotheses did't init in cpu device, make fid_dialogue_test available

中文版本

新模型推荐

高亮功能

• 支持 lora 生成扩散模型高效调优
• 增加 llama 模型
• 支持推送到 hub 的能力
• 为 chatglm-6B 类模型支持 chat 任务
• 增加常用模型和任务的 cli 调用 example

功能列表

• 支持了对使用 megatron tensor 并行模型保存的 checkpoint 拆分合并
• 支持 lora 生成扩散模型高效调优
• 增加 pedestrian attribute recognition 模型
• 增加 damofd 系列模型
• 增加 llama 模型
• 支持推送到 hub 的能力
• 增加 speaker cam++ 模型
• 增加 head 支持 XlmRoberta 模型
• 增加 canmt translation 模型
• 为 chatglm-6B 类模型支持 chat 任务

功能提升

• funasr 更新到 0.4.0 版本，支持 mac 运行
• plugin 支持 trainer
• fid_dialouge_pipeline 新增 3.7B 模型
• 增加 Mgeo 模型 token classification 任务的训练示例
• 增加 PALM 模型 text generation 任务的训练示例
• 增加 CLIP 模型 multi-modal embedding 任务的训练示例
• speech kws nearfield 训练增加梯度累积配置
• 重构优化人脸重建模型相关代码
• 更新图像着色指标
• 更新 github issue 模版

BugFix

• 修复文本生成任务模型 generate 报错
• 修复人脸重建模型 pipeline 报错
• 修复 pipeline 重复输出 warning 的问题
• 修复 plugin import 包失败时报错
• 修复 speech kws nearfield 多卡训练报错
• 修复生成模型输出英文结果缺少空格的问题
• 修复 jsonplus 不支持 ndarray 的问题

English Version

New Model List and Quick Access

Highlight

• Add efficient tunner modules
• Add llama to mslib from hf
• Support the ability to push to hub
• Add task chat for all chat models, like chatglm-6B
• Add common models and tasks cli call example

Breaking changes

Feature

• Support split and merge for megatron_base model
• Add efficient tunner modules
• Add pedestrian attribute recognition model
• Add damofd model
• Add llama to mslib from hf
• Support the ability to push to hub
• Add speaker model cam++ for speaker verification task
• New head support for XlmRoberta model
• Add canmt translation model
• Add task chat for all chat models, like chatglm-6B

Improvements

• support funasr for mac
• Plugin support trainer
• Add 3.7B size model for fid_dialouge_pipeline
• Add token classification example for MGeo
• Add PALM finetune example
• Add multi-modal embedding example for CLIP
• Speech kws nearfield training add gradient accumulation config
• Update face reconstruction to HRN(CVPR2023)
• Update image colorization metric
• Update issue templates

BugFix

• Fix generate for ModelForTextGeneration
• Fix issues for face pipeline
• Fix keep printing warnings in pipeline
• Bug fixed in plugin
• Fix speech kws nearfield training with multi-gpu
• Fix english words without space
• Fix jsonplus, support ndarray