rwkv-architecture
FeaturedRNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.
Install
Quality Score: 94/100
Skill Content
Details
- Author
- Orchestra-Research
- Repository
- Orchestra-Research/AI-Research-SKILLs
- Created
- 1 years ago
- Last Updated
- 1 months ago
- Language
- TeX
- License
- MIT
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
rwkv-architecture
RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.
rwkv-architecture
RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.
ml-transformer-era
深度时代的架构选型决策。当用户问"该用 CNN/RNN 还是 Transformer"、注意力机制/QKV 是什么、 位置编码有什么用、"数据够不够训",或想"全都上 Transformer"时激活。按数据形态×数据规模× 算力三轴匹配架构:小表格数据 GBDT 常胜神经网络、小数据 CNN/RNN 仍是正解、大规模序列才轮到 Transformer;含 QKV 直觉与位置编码必要性。不适用于:训练不动(ml-deep-training-playbook)、 自训还是微调未定(ml-pretraining-paradigm)。触发词: transformer vs cnn, QKV, positional encoding, 架构选型