aloha

mm-cot

Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tune...

CogVLM

a state-of-the-art-level open visual language model | 多模态预训练模型

open-instruct

less-is-more

Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective (ACL 2024)

STT

A multi-task model which does image captioning, sentence paraphrasing and cross-modal retrieval.

caption-by-committee

Using LLMs and pre-trained caption models for super-human performance on image captioning.

MoA

Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models

pretraining-with-human-feedback

Code accompanying the paper Pretraining Language Models with Human Preferences

CoCa-pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

OFA

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities...

awesome-foundation-and-multimodal-models

👁️ + 💬 + 🎧 = 🤖 Curated list of top foundation and multimodal models! [Paper + Code + Examples + ...

ImageCaptioning.pytorch

I decide to sync up this repo and self-critical.pytorch. (The old master is in old master branch ...

clair

CLAIR: A (surprisingly) simple semantic text metric with large language models.

xmodaler

X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image capt...

POPE

[EMNLP'23] The official GitHub page for ''Evaluating Object Hallucination in Large Vision-Languag...