Back to explore

Jev互換の公開API:SGLangによる高速並列System One生成

TypeSafe AIに触発され、著者はJev互換の公開APIを公開。同程度のオープンモデルQwen3.6-35B-A3Bを実行し、SGLang radix cacheでプレフィル再利用と高速並列System One生成を実現、64タスクを1秒未満で完了。

Inspired by @typesafeai , here is a Jev-compatible public API to play with It runs a comparable open model (Qwen3.6-35B-A3B), and just uses SGLang radix cache to preserve the prefill reuse / really fast parallel systemone generation - 64 tasks in <1s. https://github.com/ekzhang/openjev-sglang…

· 977 likesOpen on X