FastChat

Highlights

• Added SGLang worker for vision language models, lower latency and higher throughput https://github.com/lm-sys/FastChat/pull/2928
• Vision langauge WebUI https://github.com/lm-sys/FastChat/pull/2960
• OpenAI-compatible API server now supports image input https://github.com/lm-sys/FastChat/pull/2928
• Added LightLLM worker for higher throughput https://github.com/lm-sys/FastChat/blob/main/docs/lightllm_integration.md
• Added Apple MLX worker https://github.com/lm-sys/FastChat/pull/2940

What's Changed

• fix specify local path issue use model from www.modelscope.cn by @liuyhwangyh in https://github.com/lm-sys/FastChat/pull/2934
• support openai embedding for topic clustering by @CodingWithTim in https://github.com/lm-sys/FastChat/pull/2729
• Remove duplicate API endpoint by @surak in https://github.com/lm-sys/FastChat/pull/2949
• Update Hermes Mixtral by @teknium1 in https://github.com/lm-sys/FastChat/pull/2938
• Enablement of REST API Usage within Google Colab Free Tier by @ggcr in https://github.com/lm-sys/FastChat/pull/2940
• Create a new worker implementation for Apple MLX by @aliasaria in https://github.com/lm-sys/FastChat/pull/2937
• feat: support Model Yuan2.0, a new generation Fundamental Large Language Model developed by IEIT System by @cauwulixuan in https://github.com/lm-sys/FastChat/pull/2936
• Fix the pooling method of BGE embedding model by @staoxiao in https://github.com/lm-sys/FastChat/pull/2926
• SGLang Worker by @BabyChouSr in https://github.com/lm-sys/FastChat/pull/2928
• Update mlx_worker to be async by @aliasaria in https://github.com/lm-sys/FastChat/pull/2958
• Integrate LightLLM into serve worker by @zeyugao in https://github.com/lm-sys/FastChat/pull/2888
• Copy button by @surak in https://github.com/lm-sys/FastChat/pull/2963
• feat: train with template by @congchan in https://github.com/lm-sys/FastChat/pull/2951
• fix content maybe a str by @zhouzaida in https://github.com/lm-sys/FastChat/pull/2968
• Adding download folder information in README by @dheeraj-326 in https://github.com/lm-sys/FastChat/pull/2972
• use cl100k_base as the default tiktoken encoding by @bjwswang in https://github.com/lm-sys/FastChat/pull/2974
• Update README.md by @merrymercy in https://github.com/lm-sys/FastChat/pull/2975
• Fix tokenizer for vllm worker by @Michaelvll in https://github.com/lm-sys/FastChat/pull/2984
• update yuan2.0 generation by @wangpengfei1013 in https://github.com/lm-sys/FastChat/pull/2989
• fix: tokenization mismatch when training with different templates by @congchan in https://github.com/lm-sys/FastChat/pull/2996
• fix: inconsistent tokenization by llama tokenizer by @congchan in https://github.com/lm-sys/FastChat/pull/3006
• Fix type hint for play_a_match_single by @MonkeyLeeT in https://github.com/lm-sys/FastChat/pull/3008
• code update by @infwinston in https://github.com/lm-sys/FastChat/pull/2997
• Update model_support.md by @infwinston in https://github.com/lm-sys/FastChat/pull/3016
• Update lightllm_integration.md by @eltociear in https://github.com/lm-sys/FastChat/pull/3014
• Upgrade gradio to 4.17 by @infwinston in https://github.com/lm-sys/FastChat/pull/3027
• Update MLX integration to use new generate_step function signature by @aliasaria in https://github.com/lm-sys/FastChat/pull/3021
• Update readme by @merrymercy in https://github.com/lm-sys/FastChat/pull/3028
• Update gradio version in pyproject.toml and fix a bug by @merrymercy in https://github.com/lm-sys/FastChat/pull/3029
• Update gradio demo and API model providers by @merrymercy in https://github.com/lm-sys/FastChat/pull/3030
• Gradio Web Server for Multimodal Models by @BabyChouSr in https://github.com/lm-sys/FastChat/pull/2960
• Migrate the gradio server to openai v1 by @merrymercy in https://github.com/lm-sys/FastChat/pull/3032
• Update version to 0.2.36 by @merrymercy in https://github.com/lm-sys/FastChat/pull/3033

New Contributors

• @teknium1 made their first contribution in https://github.com/lm-sys/FastChat/pull/2938
• @ggcr made their first contribution in https://github.com/lm-sys/FastChat/pull/2940
• @aliasaria made their first contribution in https://github.com/lm-sys/FastChat/pull/2937
• @cauwulixuan made their first contribution in https://github.com/lm-sys/FastChat/pull/2936
• @staoxiao made their first contribution in https://github.com/lm-sys/FastChat/pull/2926
• @zhouzaida made their first contribution in https://github.com/lm-sys/FastChat/pull/2968
• @dheeraj-326 made their first contribution in https://github.com/lm-sys/FastChat/pull/2972
• @bjwswang made their first contribution in https://github.com/lm-sys/FastChat/pull/2974
• @MonkeyLeeT made their first contribution in https://github.com/lm-sys/FastChat/pull/3008

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.35...v0.2.36

What's Changed

• add dolphin by @infwinston in https://github.com/lm-sys/FastChat/pull/2794
• Fix tiny typo by @bofenghuang in https://github.com/lm-sys/FastChat/pull/2805
• Add instructions for evaluating on MT bench using vLLM by @iojw in https://github.com/lm-sys/FastChat/pull/2770
• fix missing op | for py3.8 by @dumpmemory in https://github.com/lm-sys/FastChat/pull/2800
• Add SOLAR-10.7b Instruct Model by @BabyChouSr in https://github.com/lm-sys/FastChat/pull/2826
• Update README.md by @eltociear in https://github.com/lm-sys/FastChat/pull/2852
• fix: 'compeletion' typo by @congchan in https://github.com/lm-sys/FastChat/pull/2847
• Add Tunnelmole as an open source alternative to ngrok and include usage instructions by @robbie-cahill in https://github.com/lm-sys/FastChat/pull/2846
• Add support for CatPPT by @rishiraj in https://github.com/lm-sys/FastChat/pull/2840
• Add functionality to ping AI2 InferD endpoints for tulu 2 by @natolambert in https://github.com/lm-sys/FastChat/pull/2832
• add download models from www.modelscope.cn by @liuyhwangyh in https://github.com/lm-sys/FastChat/pull/2830
• Fix conv_template of chinese alpaca 2 by @zollty in https://github.com/lm-sys/FastChat/pull/2812
• add bagel model adapter by @jondurbin in https://github.com/lm-sys/FastChat/pull/2814
• add root_path argument to gradio web server. by @stephanbertl in https://github.com/lm-sys/FastChat/pull/2807
• Import accelerate locally to avoid it as a strong dependency by @chiragjn in https://github.com/lm-sys/FastChat/pull/2820
• Replace dict merge with unpacking for compatibility of 3.8 in vLLM worker by @rudeigerc in https://github.com/lm-sys/FastChat/pull/2824
• Format code by @merrymercy in https://github.com/lm-sys/FastChat/pull/2854
• Openai API migrate by @andy-yang-1 in https://github.com/lm-sys/FastChat/pull/2765
• Add new models (Perplexity, gemini) & Separate GPT versions by @merrymercy in https://github.com/lm-sys/FastChat/pull/2856
• Clean error messages by @merrymercy in https://github.com/lm-sys/FastChat/pull/2857
• Update docs by @Ying1123 in https://github.com/lm-sys/FastChat/pull/2858
• Modify doc description by @zhangsibo1129 in https://github.com/lm-sys/FastChat/pull/2859
• Fix the problem of not using the decoding method corresponding to the base model in peft mode by @Jingsong-Yan in https://github.com/lm-sys/FastChat/pull/2865
• update a new sota model on MT-Bench which touch an 8.8 scores. by @xiechengmude in https://github.com/lm-sys/FastChat/pull/2864
• NPU needs to be initialized when starting a new process by @jq460494839 in https://github.com/lm-sys/FastChat/pull/2843
• Fix the problem with "vllm + chatglm3" (#2845) by @yaofeng in https://github.com/lm-sys/FastChat/pull/2876
• Update token spacing for mistral conversation.py by @thavens in https://github.com/lm-sys/FastChat/pull/2872
• check if hm in models before deleting to avoid errors by @joshua-ne in https://github.com/lm-sys/FastChat/pull/2870
• Add TinyLlama by @Gk-rohan in https://github.com/lm-sys/FastChat/pull/2889
• Fix bug that model doesn't automatically switch peft adapter by @Jingsong-Yan in https://github.com/lm-sys/FastChat/pull/2884
• Update web server commands by @merrymercy in https://github.com/lm-sys/FastChat/pull/2869
• fix the tokenize process and prompt template of chatglm3 by @WHDY in https://github.com/lm-sys/FastChat/pull/2883
• Add Notus support by @gabrielmbmb in https://github.com/lm-sys/FastChat/pull/2813
• feat: support anthropic api with api_dict by @congchan in https://github.com/lm-sys/FastChat/pull/2879
• Update model_adapter.py by @thavens in https://github.com/lm-sys/FastChat/pull/2895
• leaderboard code update by @infwinston in https://github.com/lm-sys/FastChat/pull/2867
• fix: change order of SEQUENCE_LENGTH_KEYS by @congchan in https://github.com/lm-sys/FastChat/pull/2925
• fix baichuan:apply_prompt_template call args error by @Force1ess in https://github.com/lm-sys/FastChat/pull/2921
• Fix a typo in openai_api_server.py by @jklj077 in https://github.com/lm-sys/FastChat/pull/2905
• feat: use variables OPENAI_MODEL_LIST by @congchan in https://github.com/lm-sys/FastChat/pull/2907
• Add TenyxChat-7B-v1 model by @sarath-shekkizhar in https://github.com/lm-sys/FastChat/pull/2901
• add support for iei yuan2.0 (https://huggingface.co/IEITYuan) by @wangpengfei1013 in https://github.com/lm-sys/FastChat/pull/2919
• nous-hermes-2-mixtral-dpo by @152334H in https://github.com/lm-sys/FastChat/pull/2922
• Bump the version to 0.2.35 by @merrymercy in https://github.com/lm-sys/FastChat/pull/2927

New Contributors

• @dumpmemory made their first contribution in https://github.com/lm-sys/FastChat/pull/2800
• @robbie-cahill made their first contribution in https://github.com/lm-sys/FastChat/pull/2846
• @rishiraj made their first contribution in https://github.com/lm-sys/FastChat/pull/2840
• @natolambert made their first contribution in https://github.com/lm-sys/FastChat/pull/2832
• @liuyhwangyh made their first contribution in https://github.com/lm-sys/FastChat/pull/2830
• @stephanbertl made their first contribution in https://github.com/lm-sys/FastChat/pull/2807
• @chiragjn made their first contribution in https://github.com/lm-sys/FastChat/pull/2820
• @rudeigerc made their first contribution in https://github.com/lm-sys/FastChat/pull/2824
• @jq460494839 made their first contribution in https://github.com/lm-sys/FastChat/pull/2843
• @yaofeng made their first contribution in https://github.com/lm-sys/FastChat/pull/2876
• @thavens made their first contribution in https://github.com/lm-sys/FastChat/pull/2872
• @joshua-ne made their first contribution in https://github.com/lm-sys/FastChat/pull/2870
• @WHDY made their first contribution in https://github.com/lm-sys/FastChat/pull/2883
• @gabrielmbmb made their first contribution in https://github.com/lm-sys/FastChat/pull/2813
• @jklj077 made their first contribution in https://github.com/lm-sys/FastChat/pull/2905
• @sarath-shekkizhar made their first contribution in https://github.com/lm-sys/FastChat/pull/2901
• @wangpengfei1013 made their first contribution in https://github.com/lm-sys/FastChat/pull/2919

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.34...v0.2.35

What's Changed

• fix tokenizer.pad_token attribute error by @wangshuai09 in https://github.com/lm-sys/FastChat/pull/2710
• support stable-vicuna model by @hi-jin in https://github.com/lm-sys/FastChat/pull/2696
• Exllama cache 8bit by @mjkaye in https://github.com/lm-sys/FastChat/pull/2719
• Add Yi support by @infwinston in https://github.com/lm-sys/FastChat/pull/2723
• Add Hermes 2.5 [fixed] by @152334H in https://github.com/lm-sys/FastChat/pull/2725
• Fix Hermes2Adapter by @lewtun in https://github.com/lm-sys/FastChat/pull/2727
• Fix YiAdapter by @Jingsong-Yan in https://github.com/lm-sys/FastChat/pull/2730
• add trust_remote_code argument by @wangshuai09 in https://github.com/lm-sys/FastChat/pull/2715
• Add revision arg to MT Bench answer generation by @lewtun in https://github.com/lm-sys/FastChat/pull/2728
• Fix MPS backend 'index out of range' error by @suquark in https://github.com/lm-sys/FastChat/pull/2737
• add starling support by @infwinston in https://github.com/lm-sys/FastChat/pull/2738
• Add deepseek chat by @BabyChouSr in https://github.com/lm-sys/FastChat/pull/2760
• a convenient script for spinning up the API with Model Workers by @ckgresla in https://github.com/lm-sys/FastChat/pull/2790
• Prevent returning partial stop string in vllm worker by @pandada8 in https://github.com/lm-sys/FastChat/pull/2780
• Update UI and new models by @infwinston in https://github.com/lm-sys/FastChat/pull/2762
• Support MetaMath by @iojw in https://github.com/lm-sys/FastChat/pull/2748
• Use common logging code in the OpenAI API server by @geekoftheweek in https://github.com/lm-sys/FastChat/pull/2758
• Show how to turn on experiment tracking for fine-tuning by @morganmcg1 in https://github.com/lm-sys/FastChat/pull/2742
• Support xDAN-L1-Chat Model by @xiechengmude in https://github.com/lm-sys/FastChat/pull/2732
• Update the version to 0.2.34 by @merrymercy in https://github.com/lm-sys/FastChat/pull/2793

New Contributors

• @mjkaye made their first contribution in https://github.com/lm-sys/FastChat/pull/2719
• @152334H made their first contribution in https://github.com/lm-sys/FastChat/pull/2725
• @Jingsong-Yan made their first contribution in https://github.com/lm-sys/FastChat/pull/2730
• @ckgresla made their first contribution in https://github.com/lm-sys/FastChat/pull/2790
• @pandada8 made their first contribution in https://github.com/lm-sys/FastChat/pull/2780
• @iojw made their first contribution in https://github.com/lm-sys/FastChat/pull/2748
• @geekoftheweek made their first contribution in https://github.com/lm-sys/FastChat/pull/2758
• @morganmcg1 made their first contribution in https://github.com/lm-sys/FastChat/pull/2742
• @xiechengmude made their first contribution in https://github.com/lm-sys/FastChat/pull/2732

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.33...v0.2.34

What's Changed

• fix: Fix for OpenOrcaAdapter to return correct conversation template by @vjsrinath in https://github.com/lm-sys/FastChat/pull/2613
• Make fastchat.serve.model_worker to take debug argument by @uinone in https://github.com/lm-sys/FastChat/pull/2628
• openchat 3.5 model support by @imoneoi in https://github.com/lm-sys/FastChat/pull/2638
• xFastTransformer framework support by @a3213105 in https://github.com/lm-sys/FastChat/pull/2615
• feat: support custom models vllm serving by @congchan in https://github.com/lm-sys/FastChat/pull/2635
• kill only fastchat process by @scenaristeur in https://github.com/lm-sys/FastChat/pull/2641
• Use conv.update_last_message api in mt-bench answer generation by @merrymercy in https://github.com/lm-sys/FastChat/pull/2647
• Improve Azure OpenAI interface by @infwinston in https://github.com/lm-sys/FastChat/pull/2651
• Add required_temp support in jsonl format to support flexible temperature setting for gen_api_answer by @CodingWithTim in https://github.com/lm-sys/FastChat/pull/2653
• Pin openai version < 1 by @infwinston in https://github.com/lm-sys/FastChat/pull/2658
• Remove exclude_unset parameter by @snapshotpl in https://github.com/lm-sys/FastChat/pull/2654
• Revert "Remove exclude_unset parameter" by @merrymercy in https://github.com/lm-sys/FastChat/pull/2666
• added support for CodeGeex(2) by @peterwilli in https://github.com/lm-sys/FastChat/pull/2645
• add chatglm3 conv template support in conversation.py by @ZeyuTeng96 in https://github.com/lm-sys/FastChat/pull/2622
• UI and model change by @infwinston in https://github.com/lm-sys/FastChat/pull/2672
• train_flant5: fix typo by @Force1ess in https://github.com/lm-sys/FastChat/pull/2673
• Fix gpt template by @infwinston in https://github.com/lm-sys/FastChat/pull/2674
• Update README.md by @merrymercy in https://github.com/lm-sys/FastChat/pull/2679
• feat: support template's stop_str as list by @congchan in https://github.com/lm-sys/FastChat/pull/2678
• Update exllama_v2.md by @jm23jeffmorgan in https://github.com/lm-sys/FastChat/pull/2680
• save model under deepspeed by @MrZhengXin in https://github.com/lm-sys/FastChat/pull/2689
• Adding SSL support for model workers and huggingface worker by @lnguyen in https://github.com/lm-sys/FastChat/pull/2687
• Check the max_new_tokens <= 0 in openai api server by @zeyugao in https://github.com/lm-sys/FastChat/pull/2688
• Add Microsoft/Orca-2-7b and update model support docs by @BabyChouSr in https://github.com/lm-sys/FastChat/pull/2714
• fix tokenizer of chatglm2 by @wangshuai09 in https://github.com/lm-sys/FastChat/pull/2711
• Template for using Deepseek code models by @AmaleshV in https://github.com/lm-sys/FastChat/pull/2705
• add support for Chinese-LLaMA-Alpaca by @zollty in https://github.com/lm-sys/FastChat/pull/2700
• Make --load-8bit flag work with weights in safetensors format by @xuguodong1999 in https://github.com/lm-sys/FastChat/pull/2698
• Format code and minor bug fix by @merrymercy in https://github.com/lm-sys/FastChat/pull/2716
• Bump version to v0.2.33 by @merrymercy in https://github.com/lm-sys/FastChat/pull/2717

New Contributors

• @vjsrinath made their first contribution in https://github.com/lm-sys/FastChat/pull/2613
• @uinone made their first contribution in https://github.com/lm-sys/FastChat/pull/2628
• @a3213105 made their first contribution in https://github.com/lm-sys/FastChat/pull/2615
• @scenaristeur made their first contribution in https://github.com/lm-sys/FastChat/pull/2641
• @snapshotpl made their first contribution in https://github.com/lm-sys/FastChat/pull/2654
• @peterwilli made their first contribution in https://github.com/lm-sys/FastChat/pull/2645
• @ZeyuTeng96 made their first contribution in https://github.com/lm-sys/FastChat/pull/2622
• @Force1ess made their first contribution in https://github.com/lm-sys/FastChat/pull/2673
• @jm23jeffmorgan made their first contribution in https://github.com/lm-sys/FastChat/pull/2680
• @MrZhengXin made their first contribution in https://github.com/lm-sys/FastChat/pull/2689
• @lnguyen made their first contribution in https://github.com/lm-sys/FastChat/pull/2687
• @wangshuai09 made their first contribution in https://github.com/lm-sys/FastChat/pull/2711
• @AmaleshV made their first contribution in https://github.com/lm-sys/FastChat/pull/2705
• @zollty made their first contribution in https://github.com/lm-sys/FastChat/pull/2700
• @xuguodong1999 made their first contribution in https://github.com/lm-sys/FastChat/pull/2698

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.32...v0.2.33

What's Changed

• Fix for single turn dataset by @toslunar in https://github.com/lm-sys/FastChat/pull/2509
• replace os.getenv with os.path.expanduser because the first one doesn… by @khalil-Hennara in https://github.com/lm-sys/FastChat/pull/2515
• Fix arena by @merrymercy in https://github.com/lm-sys/FastChat/pull/2522
• Update Dockerfile by @dubaoquan404 in https://github.com/lm-sys/FastChat/pull/2524
• add Llama2ChangAdapter by @lcw99 in https://github.com/lm-sys/FastChat/pull/2510
• Add ExllamaV2 Inference Framework Support. by @leonxia1018 in https://github.com/lm-sys/FastChat/pull/2455
• Improve docs by @merrymercy in https://github.com/lm-sys/FastChat/pull/2534
• Fix warnings for new gradio versions by @merrymercy in https://github.com/lm-sys/FastChat/pull/2538
• Improve chat templates by @merrymercy in https://github.com/lm-sys/FastChat/pull/2539
• Add Zephyr 7B Alpha by @lewtun in https://github.com/lm-sys/FastChat/pull/2535
• Improve Support for Mistral-Instruct by @Steve-Tech in https://github.com/lm-sys/FastChat/pull/2547
• correct max_tokens by context_length instead of raise exception by @liunux4odoo in https://github.com/lm-sys/FastChat/pull/2544
• Revert "Improve Support for Mistral-Instruct" by @merrymercy in https://github.com/lm-sys/FastChat/pull/2552
• Fix Mistral template by @normster in https://github.com/lm-sys/FastChat/pull/2529
• Add additional Informations from the vllm worker by @SebastianBodza in https://github.com/lm-sys/FastChat/pull/2550
• Make FastChat work with LMSYS-Chat-1M Code by @CodingWithTim in https://github.com/lm-sys/FastChat/pull/2551
• Create tags attribute to fix MarkupError in rich CLI by @Steve-Tech in https://github.com/lm-sys/FastChat/pull/2553
• move BaseModelWorker outside serve.model_worker to make it independent by @liunux4odoo in https://github.com/lm-sys/FastChat/pull/2531
• Misc style and bug fixes by @merrymercy in https://github.com/lm-sys/FastChat/pull/2559
• Fix README.md by @infwinston in https://github.com/lm-sys/FastChat/pull/2561
• release v0.2.31 by @merrymercy in https://github.com/lm-sys/FastChat/pull/2563
• resolves #2542 modify dockerfile to upgrade cuda to 12.2.0 and pydantic 1.10.13 by @alexdelapaz in https://github.com/lm-sys/FastChat/pull/2565
• Add airoboros_v3 chat template (llama-2 format) by @jondurbin in https://github.com/lm-sys/FastChat/pull/2564
• Add Xwin-LM V0.1, V0.2 support by @REIGN12 in https://github.com/lm-sys/FastChat/pull/2566
• Fixed model_worker generate_gate may blocked main thread (#2540) by @lvxuan263 in https://github.com/lm-sys/FastChat/pull/2562
• feat: add claude-v2 by @congchan in https://github.com/lm-sys/FastChat/pull/2571
• Update vigogne template by @bofenghuang in https://github.com/lm-sys/FastChat/pull/2580
• Fix issue #2568: --device mps led to TypeError: forward() got an unexpected keyword argument 'padding_mask'. by @Phil-U-U in https://github.com/lm-sys/FastChat/pull/2579
• Add Mistral-7B-OpenOrca conversation_temmplate by @waynespa in https://github.com/lm-sys/FastChat/pull/2585
• docs: bit misspell comments model adapter default template name conversation by @guspan-tanadi in https://github.com/lm-sys/FastChat/pull/2594
• Update Mistral template by @Gk-rohan in https://github.com/lm-sys/FastChat/pull/2581
• Update README.md (vicuna-v1.3 -> vicuna-1.5) by @infwinston in https://github.com/lm-sys/FastChat/pull/2592
• Update README.md to highlight chatbot arena by @infwinston in https://github.com/lm-sys/FastChat/pull/2596
• Add Lemur model by @ugolotti in https://github.com/lm-sys/FastChat/pull/2584
• add trust_remote_code=True in BaseModelAdapter by @edisonwd in https://github.com/lm-sys/FastChat/pull/2583
• Openai interface add use beam search and best of 2 by @leiwen83 in https://github.com/lm-sys/FastChat/pull/2442
• Update qwen and add pygmalion by @Trangle in https://github.com/lm-sys/FastChat/pull/2607
• feat: Support model AquilaChat2 by @fangyinc in https://github.com/lm-sys/FastChat/pull/2616
• Added settings vllm by @SebastianBodza in https://github.com/lm-sys/FastChat/pull/2599
• [Logprobs] Support logprobs=1 by @comaniac in https://github.com/lm-sys/FastChat/pull/2612

New Contributors

• @toslunar made their first contribution in https://github.com/lm-sys/FastChat/pull/2509
• @khalil-Hennara made their first contribution in https://github.com/lm-sys/FastChat/pull/2515
• @dubaoquan404 made their first contribution in https://github.com/lm-sys/FastChat/pull/2524
• @leonxia1018 made their first contribution in https://github.com/lm-sys/FastChat/pull/2455
• @lewtun made their first contribution in https://github.com/lm-sys/FastChat/pull/2535
• @normster made their first contribution in https://github.com/lm-sys/FastChat/pull/2529
• @SebastianBodza made their first contribution in https://github.com/lm-sys/FastChat/pull/2550
• @alexdelapaz made their first contribution in https://github.com/lm-sys/FastChat/pull/2565
• @REIGN12 made their first contribution in https://github.com/lm-sys/FastChat/pull/2566
• @lvxuan263 made their first contribution in https://github.com/lm-sys/FastChat/pull/2562
• @Phil-U-U made their first contribution in https://github.com/lm-sys/FastChat/pull/2579
• @waynespa made their first contribution in https://github.com/lm-sys/FastChat/pull/2585
• @guspan-tanadi made their first contribution in https://github.com/lm-sys/FastChat/pull/2594
• @Gk-rohan made their first contribution in https://github.com/lm-sys/FastChat/pull/2581
• @ugolotti made their first contribution in https://github.com/lm-sys/FastChat/pull/2584
• @edisonwd made their first contribution in https://github.com/lm-sys/FastChat/pull/2583
• @fangyinc made their first contribution in https://github.com/lm-sys/FastChat/pull/2616
• @comaniac made their first contribution in https://github.com/lm-sys/FastChat/pull/2612

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.30...v0.2.32

What's Changed

• Support new models
• Bug fixes

New Contributors

• @wangxiyuan made their first contribution in https://github.com/lm-sys/FastChat/pull/2404
• @wangzhen263 made their first contribution in https://github.com/lm-sys/FastChat/pull/2402
• @karshPrime made their first contribution in https://github.com/lm-sys/FastChat/pull/2406
• @obitolyz made their first contribution in https://github.com/lm-sys/FastChat/pull/2408
• @Somezak1 made their first contribution in https://github.com/lm-sys/FastChat/pull/2431
• @hi-jin made their first contribution in https://github.com/lm-sys/FastChat/pull/2434
• @zhangsibo1129 made their first contribution in https://github.com/lm-sys/FastChat/pull/2422
• @tobiabir made their first contribution in https://github.com/lm-sys/FastChat/pull/2418
• @Btlmd made their first contribution in https://github.com/lm-sys/FastChat/pull/2384
• @brandonbiggs made their first contribution in https://github.com/lm-sys/FastChat/pull/2448
• @dongxiaolong made their first contribution in https://github.com/lm-sys/FastChat/pull/2463
• @shuishu made their first contribution in https://github.com/lm-sys/FastChat/pull/2482
• @asaiacai made their first contribution in https://github.com/lm-sys/FastChat/pull/2469
• @hnyls2002 made their first contribution in https://github.com/lm-sys/FastChat/pull/2456
• @enochlev made their first contribution in https://github.com/lm-sys/FastChat/pull/2499
• @AlpinDale made their first contribution in https://github.com/lm-sys/FastChat/pull/2500
• @lerela made their first contribution in https://github.com/lm-sys/FastChat/pull/2483

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.28...v0.2.30

What's Changed

• Multiple UI updates, performance improvements and bug fixes
• New model support (Spicyboros + airoboros 2.2, VMware's OpenLLaMa OpenInstruct)
• Add sponsors (Kaggle, MBZUAI, AnyScale, and Huggingface)

New Contributors

• @nicobasile made their first contribution in https://github.com/lm-sys/FastChat/pull/2278
• @zeyugao made their first contribution in https://github.com/lm-sys/FastChat/pull/2297
• @fan-chao made their first contribution in https://github.com/lm-sys/FastChat/pull/2290
• @leiwen83 made their first contribution in https://github.com/lm-sys/FastChat/pull/2273
• @siddartha-RE made their first contribution in https://github.com/lm-sys/FastChat/pull/2263
• @renatz made their first contribution in https://github.com/lm-sys/FastChat/pull/2296
• @so2liu made their first contribution in https://github.com/lm-sys/FastChat/pull/2225
• @epec254 made their first contribution in https://github.com/lm-sys/FastChat/pull/2306
• @woshiyyya made their first contribution in https://github.com/lm-sys/FastChat/pull/2326
• @vaxilicaihouxian made their first contribution in https://github.com/lm-sys/FastChat/pull/2328
• @nathanstitt made their first contribution in https://github.com/lm-sys/FastChat/pull/2337

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.25...v0.2.28

What's Changed

• Support new models (Qwen, WizardCoder, Llama2-Chinese, BAAI/AquilaChat-7B, OpenOrca, BGE)
• Improve performance and usability. Fix bugs.
• Reduce dependency by making some required packages optional

New Contributors

• @azshue made their first contribution in https://github.com/lm-sys/FastChat/pull/2169
• @liunux4odoo made their first contribution in https://github.com/lm-sys/FastChat/pull/2147
• @tmm1 made their first contribution in https://github.com/lm-sys/FastChat/pull/2126
• @shibing624 made their first contribution in https://github.com/lm-sys/FastChat/pull/2138
• @Tomorrowxxy made their first contribution in https://github.com/lm-sys/FastChat/pull/2185
• @Extremys made their first contribution in https://github.com/lm-sys/FastChat/pull/2166
• @gesanqiu made their first contribution in https://github.com/lm-sys/FastChat/pull/2192
• @alongLFB made their first contribution in https://github.com/lm-sys/FastChat/pull/2202
• @congchan made their first contribution in https://github.com/lm-sys/FastChat/pull/2194
• @Rayrtfr made their first contribution in https://github.com/lm-sys/FastChat/pull/2218
• @bofenghuang made their first contribution in https://github.com/lm-sys/FastChat/pull/2236
• @Cyrilvallez made their first contribution in https://github.com/lm-sys/FastChat/pull/2239
• @persistz made their first contribution in https://github.com/lm-sys/FastChat/pull/2248
• @LeiZhou-97 made their first contribution in https://github.com/lm-sys/FastChat/pull/2247

Full Changelog: https://github.com/lm-sys/FastChat/compare/v0.2.23...v0.2.25

• Released Vicuna v1.5 based on Llama 2 with 4K and 16K context lengths. Download weights
• Released Chatbot Arena Conversations, a dataset containing 33k conversations with human preferences. Download it here.
• Serving
  • Add a multi-model worker that can host multiple models on a single GPU and share base weights for PEFT models. #1866 #1905
  • AWQ 4-bit quantization support. #2103
  • Support model models (Llama 2, Claude 2, ChatGLM 2, StarChat, Baichuan-13B, InternLM, airoboros, PEFT adapters).
  • Better support for AMD GPUs, Intel XPUs. #1954 #2052
• Training
  • Support rope scaling. #2013
  • Support flash attention 2. #2059
  • Support xformer. #1970

• Release MT-bench code and data
  • Demo, Leaderboard, Human annotation data
• Release new models
  • Vicuna-33B v1.3 and LongChat
• Support more models (Falcon, Salesforce/xgen, Salesforce/codet5p-6b, Robin-7B/13B/33B, Baichuan-7B)
• Integrate vLLM worker for continuous batching and high-throughput serving. See doc.

• LangChain integration with OpenAI-compatible API server (doc)
• Enhanced OpenAI-compatible API server (doc)
• More model support (https://chat.lmsys.org/?arena)
  • Guanaco, WizardLM, GPT4ALL, and more.
• Bug fixes

• Added OpenAI-compatible API server (https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md). Now you can use FastChat as a local drop-in replacement for OpenAI APIs.
• Supported Bard and Claude. Now Google's Bard and Anthropic's Claude are available at arena.lmsys.org
• Supported various open-source models (e.g., MPT-Chat-7B, Phoenix-inst-chat-7b)
• Other bug fixes

• Released FastChat-T5 compatible with commercial usage. Checkout weights.
• Announced ChatBot Arena for comparing open LLMs. Checkout demo.
• Increased the model support coverage. The following models are tested with FastChat CLI and GUI.
  • Vicuna, Alpaca, LLaMA, Koala
  • lmsys/fastchat-t5-3b-v1.0
  • BlinkDL/RWKV-4-Raven
  • databricks/dolly-v2-12b
  • OpenAssistant/oasst-sft-1-pythia-12b
  • project-baize/baize-lora-7B
  • StabilityAI/stablelm-tuned-alpha-7b
  • THUDM/chatglm-6b

• Added a Chatbot Arena where you can compare open large models side-by-side (https://arena.lmsys.org/)
  • Now we support Vicuna, Koala, OpenAssistant, Dolly, ChatGLM, StableLM, Alpaca, LLaMA
• Fix bugs