All 11 of the pre-trained checkpoints are available on the Hugging Face Hub. The checkpoints are summarised in the following table with links to the models on the Hub:
Size | Layers | Width | Heads | Parameters | English-only | Multilingual |
---|---|---|---|---|---|---|
tiny | 4 | 384 | 6 | 39 M | ✓ | ✓ |
base | 6 | 512 | 8 | 74 M | ✓ | ✓ |
small | 12 | 768 | 12 | 244 M | ✓ | ✓ |
medium | 24 | 1024 | 16 | 769 M | ✓ | ✓ |
large | 32 | 1280 | 20 | 1550 M | x | ✓ |
large-v2 | 32 | 1280 | 20 | 1550 M | x | ✓ |
large-v3 | 32 | 1280 | 20 | 1550 M | x | ✓ |
About 1 min