RWKV-6 Finch Models by RWKV Project
Details
About
RWKV-6 (codenamed Finch) is the sixth-generation architecture in the RWKV series, introduced in the paper 'Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence' (April 2024). Finch upgrades the state representation to multi-headed matrix-valued states and introduces dynamic data-driven recurrence, achieving significantly better benchmark performance than RWKV-5 (Eagle) without sacrificing the architecture's defining O(n) training complexity and O(1) constant-memory inference. Models are available in 1.6B, 3B, 7B, and 14B parameter sizes. The 14B model (released September 2024) was created by stacking two 7B models and is the largest released RWKV-6 model. All World variants are multilingual (100+ languages). Apache 2.0 licensed.
Current Variants
Use-when guidance is based on each model's tracked capabilities, context window, release date, and replacement status.
Use when the workload needs Infinity context and 14B parameters.
Use when the workload needs Infinity context and 7B parameters.
Use when the workload needs Infinity context and 3B parameters.
Use when the workload needs Infinity context and 1.6B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| RWKV-6 Finch 14B | Use when the workload needs Infinity context and 14B parameters. | 2024-09 | Infinity context14B parameters | Current |
| RWKV-6 Finch 7B | Use when the workload needs Infinity context and 7B parameters. | 2024-04 | Infinity context7B parameters | Current |
| RWKV-6 Finch 3B | Use when the workload needs Infinity context and 3B parameters. | 2024-04 | Infinity context3B parameters | Current |
| RWKV-6 Finch 1.6B | Use when the workload needs Infinity context and 1.6B parameters. | 2024-04 | Infinity context1.6B parameters | Current |
Release Timeline
2 release groupsSpecifications(4 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| RWKV-6 Finch 14B | 2024-09 | Infinite | 14B |
| RWKV-6 Finch 7B | 2024-04 | Infinite | 7B |
| RWKV-6 Finch 3B | 2024-04 | Infinite | 3B |
| RWKV-6 Finch 1.6B | 2024-04 | Infinite | 1.6B |
Frequently Asked Questions
- What is RWKV-6 Finch used for?
- RWKV-6 Finch is used for coding. The family description and listed model capabilities point to those workloads as the best fit.
- How does RWKV-6 Finch compare to RWKV-7 Goose?
- RWKV-6 Finch by RWKV Project is strongest where you need coding, while RWKV-7 Goose by RWKV Project is the closest related family to check for coding. RWKV-6 Finch has 4 listed variants, so compare the specs and pricing tables before choosing a production model.
- Which RWKV-6 Finch model should I use?
- If price is the main constraint, use the pricing table first because RWKV-6 Finch does not have complete provider pricing in the local data. For the most capable/latest local choice, evaluate RWKV-6 Finch 14B.