Packages, Inference & Training

Inference service

The fastest pure C++ GPU runtime for RWKV — zero PyTorch, one-command deployment, CUDA/ROCm inference, quantization, state tuning & serving.🕊

Inference service

Independent SGLang distribution focused on production-grade RWKV-7 serving, recurrent state caching, speculative decoding, quantization, and broad GPU support.

Quantization and compression

RWKV-7 inference engine in Rust (Vulkan compute shaders) with fp16/any4/int8 quantization paths

Inference backend

Native RWKV-7 integration for vLLM, implemented clean-room against the Hugging Face contract.

End-user application

Cross-platform, local-first RWKV chat application for Android, iOS, Windows, macOS, and Linux.

End-user application

Flutter FFI bridge and runtime layer for on-device RWKV inference across mobile and desktop platforms.

Training and fine-tuning

The primary RWKV language-model repository, including architecture, training code, CUDA kernels, and examples.

End-user application

rwkv role play

Inference service

onnx_rwkv provides RWKV inference, deployment, runtime, or serving support.

Training and fine-tuning

Research tools for converting attention-based architectures to RWKV through layerwise distillation.

End-user application

Web frontend for high-concurrency RWKV inference.

Quantization and compression

pip install rwkv-quant

Inference backend

pip install rwkv-metal

RWKV research implementation

Chinese hands-on course for building GLM4, Llama 3, and RWKV-6 language models from scratch.

End-user application

RWKV Vela for Mac — local RWKV chat, model dashboard and signed app updates

Training and fine-tuning

QLoRA fine-tuning framework for RWKV7-G1j models with custom datasets and training scripts.

Training and fine-tuning

RWKV-7 state tuning for Mac (Apple Silicon / MLX) — train mountable init states.

Training and fine-tuning

Research on the RWKV recurrent LLM architecture: a custom fused CUDA WKV kernel, architecture refactors (SwiGLU -> linear projection), and training experiments on synthetic regular-language tasks.

Inference runtime

RWKV-7 7.2B fp16 15000+ tps decoding @ single 5090

Inference runtime

Inference for hybrid LLMs: Gemma, RWKV, and all kinds of hybrids.

Runtime adapter

RWKV-7 HuggingFace / Transformers adapter

End-user application

Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.

End-user application

Inference RWKV with multiple supported backends.

End-user application

Automated RWKV management and startup tool with a desktop interface and an OpenAI-compatible API.