Model Directory · AI Research Newsletter

Benchmarks are reported results, not a ranking. Scores from different datasets, versions, prompting, training, or evaluation setups are not directly comparable. “Not reported” means the cited sources do not state the detail; “Not available” means no linked artifact is supplied.

LLM (text → text)

Llama 3.1 8B Instruct

Release date: 2024-07-23
Organization: Meta
Parameters: 8 billion (reported model size)

Open weights
Llama 3.1 Community License Agreement

Text assistant supporting eight languages, with a 128k context window and instruction tuning for dialogue.

Reported scores; setups differ. Expand for evaluation details.

📄 Paper: The Llama 3 Herd of Models
💻 Code: Meta reference implementation

Show moreShow less

Downloadable weights; custom license and acceptable-use restrictions; gated access on Hugging Face.

Autoregressive Transformer with grouped-query attention; SFT and RLHF instruction tuning.

MMLU: 69.4%
Instruction-tuned 8B; 5-shot; macro_avg/acc; Meta internal evaluation library. Dataset revision and library version: Not reported.
Meta model card

HumanEval: 72.6%
Instruction-tuned 8B; 0-shot; pass@1; Meta internal evaluation library. Dataset revision and sampling settings: Not reported.
Meta model card

SourcesMeta model card

Multimodal LLM / VLM (image + text → text)

GPT-4 (original, 2023)

Release date: 2023-03-14
Organization: OpenAI
Parameters: Parameters: Not reported

Proprietary
No public model license; hosted service governed by OpenAI terms

Multimodal model for image and text understanding and text generation. This entry describes the original report, rather than later GPT-4 variants.

MMLU 86.4%

Reported scores; setups differ. Expand for evaluation details.

📄 Paper: Technical report, Table 2
💻 Code: Not available

Show moreShow less

No released weights. At launch: text access through ChatGPT and an API waitlist; image access limited. Historical release, not a current availability guarantee.

Transformer trained for next-token prediction and aligned with RLHF; detailed architecture: Not reported.

MMLU: 86.4%
Original 2023 technical report, Table 2; 5-shot accuracy over 57 subjects; text evaluation. Dataset revision: Not reported. Report notes minor differences from standard evaluation setups.
Technical report, Table 2

SourcesLaunch announcement
Technical report, Table 2

VLM (image–text embedding model)

CLIP ViT-B/32

Release date: 2021-01-05
Organization: OpenAI
Parameters: Parameters: Not reported

Open source
MIT License (released code and model weights)

Contrastively trained image and text encoders for retrieval and zero-shot classification using natural-language class descriptions.

ImageNet-1K 63.2% top-1

Reported scores; setups differ. Expand for evaluation details.

📄 Paper: Learning Transferable Visual Models From Natural Language Supervision
💻 Code: OpenAI CLIP

Show moreShow less

Public code and pretrained weights; training dataset is not released.

Dual encoder: ViT-B/32 image encoder and Transformer text encoder; contrastive image–text training.

ImageNet-1K: 63.2% top-1
CLIP paper Table 17; ViT-B/32, 224px input; zero-shot classification with prompt ensembling, no ImageNet classifier training. Evaluation uses ImageNet validation; dataset revision: Not reported.
CLIP paper, Table 17

SourcesRelease announcement
CLIP paper, Table 17
Official model card

Vision model (image classification)

DeiT-Base (224, non-distilled)

Release date: Not reported (paper first submitted 2020-12-23)
Organization: Meta FAIR
Parameters: 86 million (official model zoo)

Open source
Apache License 2.0 (repository release)

Image classifier trained on ImageNet without external training data. This entry is the baseline model; the paper also introduces attention-based distillation variants.

ImageNet-1K 81.8% top-1; 95.6% top-5

Reported scores; setups differ. Expand for evaluation details.

📄 Paper: Training data-efficient image transformers & distillation through attention
💻 Code: Official DeiT repository

Show moreShow less

Public code and baseline pretrained checkpoints linked in the official repository.

Vision Transformer, base size; 16×16 image patches; 224×224 input; non-distilled baseline.

ImageNet-1K: 81.8% top-1; 95.6% top-5
Official baseline DeiT-base checkpoint; ImageNet 2012 training only, no external data; 224px, single-crop validation. Reference inference uses timm 0.3.2. Distilled and 384px variants have different scores.
Official model zoo and evaluation instructions

SourcesOfficial model zoo and evaluation instructions
Paper and author affiliations

On small screens, scroll horizontally to read every field.

Model directory — scores use the evaluation setups shown
Model	Release date	Organization	License & availability	Model type	Architecture	Parameters	Benchmarks	Paper & code	Sources
Llama 3.1 8B Instruct	2024-07-23	Meta	Open weights Llama 3.1 Community License Agreement Downloadable weights; custom license and acceptable-use restrictions; gated access on Hugging Face.	LLM (text → text)	Autoregressive Transformer with grouped-query attention; SFT and RLHF instruction tuning.	8 billion (reported model size)	MMLU: 69.4% Instruction-tuned 8B; 5-shot; macro_avg/acc; Meta internal evaluation library. Dataset revision and library version: Not reported. Meta model card HumanEval: 72.6% Instruction-tuned 8B; 0-shot; pass@1; Meta internal evaluation library. Dataset revision and sampling settings: Not reported. Meta model card	📄 Paper: The Llama 3 Herd of Models 💻 Code: Meta reference implementation	Meta model card
GPT-4 (original, 2023)	2023-03-14	OpenAI	Proprietary No public model license; hosted service governed by OpenAI terms No released weights. At launch: text access through ChatGPT and an API waitlist; image access limited. Historical release, not a current availability guarantee.	Multimodal LLM / VLM (image + text → text)	Transformer trained for next-token prediction and aligned with RLHF; detailed architecture: Not reported.	Not reported	MMLU: 86.4% Original 2023 technical report, Table 2; 5-shot accuracy over 57 subjects; text evaluation. Dataset revision: Not reported. Report notes minor differences from standard evaluation setups. Technical report, Table 2	📄 Paper: Technical report, Table 2 💻 Code: Not available	Launch announcement Technical report, Table 2
CLIP ViT-B/32	2021-01-05	OpenAI	Open source MIT License (released code and model weights) Public code and pretrained weights; training dataset is not released.	VLM (image–text embedding model)	Dual encoder: ViT-B/32 image encoder and Transformer text encoder; contrastive image–text training.	Not reported	ImageNet-1K: 63.2% top-1 CLIP paper Table 17; ViT-B/32, 224px input; zero-shot classification with prompt ensembling, no ImageNet classifier training. Evaluation uses ImageNet validation; dataset revision: Not reported. CLIP paper, Table 17	📄 Paper: Learning Transferable Visual Models From Natural Language Supervision 💻 Code: OpenAI CLIP	Release announcement CLIP paper, Table 17 Official model card
DeiT-Base (224, non-distilled)	Not reported (paper first submitted 2020-12-23)	Meta FAIR	Open source Apache License 2.0 (repository release) Public code and baseline pretrained checkpoints linked in the official repository.	Vision model (image classification)	Vision Transformer, base size; 16×16 image patches; 224×224 input; non-distilled baseline.	86 million (official model zoo)	ImageNet-1K: 81.8% top-1; 95.6% top-5 Official baseline DeiT-base checkpoint; ImageNet 2012 training only, no external data; 224px, single-crop validation. Reference inference uses timm 0.3.2. Distilled and 384px variants have different scores. Official model zoo and evaluation instructions	📄 Paper: Training data-efficient image transformers & distillation through attention 💻 Code: Official DeiT repository	Official model zoo and evaluation instructions Paper and author affiliations